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Overview 



Can a multicomponent employment initiative that is located in public housing developments 
help residents work, earn more money, and improve their quality of life? The Jobs-Plus Com- 
munity Revitalization Initiative for Public Housing Families (Jobs-Plus, for short) sought to 
achieve these goals at selected public housing developments in six cities: Baltimore, Chatta- 
nooga, Dayton, Los Angeles, St. Paul, and Seattle. Jobs-Plus was conducted as a research dem- 
onstration project from 1998 to 2003 with sponsorship from a consortium of funders, led by the 
U.S. Department of Housing and Urban Development and the Rockefeller Foundation. The 
program — which was targeted to all working-age, nondisabled residents of selected public 
housing developments and implemented by a collaboration of local organizations — had three 
main components: employment-related services, rent-based work incentives that allowed resi- 
dents to keep more of their earnings, and activities to promote neighbor-to-neighbor support for 
work. This final report on MDRC’s evaluation of Jobs-Plus describes the program’s impacts, 
that is, the difference it made for residents in Jobs-Plus developments in comparison with resi- 
dents living in similar developments who did not receive the program. 



Key Findings 

• Four of the six study sites built substantial Jobs-Plus programs — although it took over two 
years to accomplish. 

• Once Jobs-Plus was in place at the four sites, it markedly increased the earnings of residents 
(including those who eventually moved away) relative to the comparison group. This im- 
pact was sustained over time at three of the sites but disappeared at the fourth when its resi- 
dents were displaced by a federal HOPE VI renovation project. There was no program ef- 
fect on earnings at the two sites that did not fully implement Jobs-Plus. The effects of Jobs- 
Plus on employment were positive at the sites that substantially implemented the program 
but were smaller and less consistent than the effects on earnings. 

• The large positive earnings effect of Jobs-Plus in the stronger implementation sites held for 
a wide range of residents defined in terms of their gender, race or ethnicity, age, past em- 
ployment, past welfare receipt, past duration of residence, and future resident mobility. 
Most striking were the especially large impacts for immigrant men. 

• Welfare receipt by residents dropped precipitously after Jobs-Plus was launched, but this 
decline was not related to Jobs-Plus. 

• The positive effects on individual earnings were more likely to translate into more earnings 
across the housing development as a whole in sites where fewer residents moved out. How- 
ever, these effects did not spark changes in overall social conditions or quality of life in the 
developments. 

These and other findings in the report offer important lessons to policymakers and program ad- 
ministrators about how to increase the economic self-sufficiency of public housing residents. 
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Preface 



Sociologist William Julius Wilson and other researchers have documented the conse- 
quences of concentrated joblessness and poverty in urban neighborhoods, conditions that are 
often especially severe among residents of public housing. With this in mind, the U.S. Depart- 
ment of Housing and Urban Development, the Rockefeller Foundation, and MDRC launched an 
ambitious enterprise called the Jobs-Plus Community Revitalization Initiative for Public Hous- 
ing Families (Jobs-Plus), a comprehensive program located within public housing developments 
and designed to help move residents into work and increase their earnings. This final report on 
the initiative offers encouraging evidence that a strategy like Jobs-Plus can help people in high- 
poverty public housing developments succeed in the labor market. 

Success was by no means assured at the outset of Jobs-Plus. Living conditions in public 
housing are difficult, and few employment programs have been attempted — let alone rigor- 
ously tested — in such settings. The challenges were myriad: developing an innovative and 
credible research design; creating real collaboration among housing authorities, social service 
agencies, and residents; appealing to residents of different racial and ethnic groups; and dealing 
with enduring issues of safety, substance abuse, and other family crises. The six study sites 
themselves, as well as the many funders who provided support, deserve great credit for their 
steadfast work and commitment through this nearly decade-long project. 

In an era of scarce resources, when policymakers seek ever more information about 
whether programs work, Jobs-Plus offers hard evidence that a work-focused intervention based 
in public housing developments can be effective. However, the report also shows that success 
requires the commitment of housing officials who see a broad mission for public housing in the 
nation’s social safety net, as well as the active partnership of the welfare and workforce sys- 
tems. Most importantly, these findings from Jobs-Plus speak directly to the challenges confront- 
ing managers of public housing, who feel increasingly pressured to promote the self-sufficiency 
of residents and change the income mix of those living in public housing. We hope that the 
promising lessons of Jobs-Plus can help policymakers and practitioners meet those challenges. 



Gordon L. Berlin 
President 
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Executive Summary 



Can a multicomponent employment initiative that is located in public housing devel- 
opments help residents work, earn more money, and improve their quality of life? The Jobs- 
Plus Community Revitalization Initiative for Public Housing Families (Jobs-Plus, for short) 
sought to achieve these ambitious goals in difficult environments. Operated as a special demon- 
stration project in selected housing developments in six U.S. cities, Jobs-Plus was sponsored by 
a consortium of public and private funders led by the U.S. Department of Housing and Urban 
Development and the Rockefeller Foundation. MDRC, a nonprofit social policy research firm, 
managed the demonstration and evaluated the program. 

This final MDRC report on the initiative assesses the program’s success in achieving 
key outcomes for residents and their housing developments. It analyzes the program’ s effects — 
or “impacts” — on residents’ employment rates, average earnings, and welfare receipt by com- 
paring the outcomes for residents of the Jobs-Plus developments with the outcomes for their 
counterparts in similar “comparison” developments that did not implement the program. (Be- 
cause housing developments were allocated randomly to the Jobs-Plus or comparison group, 
their outcomes provide an especially rigorous basis for estimating program impacts.) The report 
also examines changes in social and material conditions at the developments. 

In summary, the findings show that: 

• For all sites combined, Jobs-Plus produced positive impacts on residents’ 
earnings, whether or not the residents continued living in their developments. 

• These overall effects were driven primarily by large and sustained impacts in 
three sites (in Dayton, Los Angeles, and St. Paul) where the implementation of 
Jobs-Plus was stronger and more complete. A fourth site (Seattle) had strong 
early earnings effects that ended when residents were displaced by a federal 
Hope VI renovation project. The program had no earnings effects in two sites 
(Baltimore and Chattanooga) that did not fully implement Jobs-Plus. 

• These impacts were more likely to translate into higher earnings in the housing 
development as a whole in sites where fewer residents moved out. However, 
the program’s effects did not spark changes in broader social conditions. 

• In the stronger implementation sites, Jobs-Plus had positive earnings impacts 
for many different types of residents, striking earnings effects for immigrant 
men, positive but smaller impacts on residents’ employment rates, and no 
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impact on residents’ welfare receipt (because rates were dropping precipi- 
tously among all welfare recipients). 



These findings offer lessons to policymakers and program administrators about a num- 
ber of important issues, including the considerable willingness and ability of public housing 
residents to enter the labor market, the importance of rent-based financial incentives as a pro- 
gram “hook” and a driver of the positive earnings impacts, and the critical role of housing au- 
thority leadership in the implementation of a “place-based” self-sufficiency initiative in public 
housing. They also show the promise of one approach to achieving the employment and self- 
sufficiency objectives of the 1998 federal housing law (the Quality Housing and Work Respon- 
sibility Act, or QHWRA). 



What Is Jobs-Plus? 

Jobs-Plus attempted to deliver an employment and training program within public 
housing developments to all working-age, nondisabled residents. The initiative had three core 
components: 

• Employment-related services and activities to help residents secure and 
retain employment, including job search assistance, education programs, 
vocational training, and such support services as child care and transporta- 
tion assistance. 

• Financial incentives to work, consisting of changes in public housing rent 
rules that helped make work “pay” by reducing the extent to which increases 
in earnings were offset by increased rents. 

• Community support for work, which sought to strengthen social ties 
among residents in ways that would help support their job preparation and 
work efforts — for instance, by fostering neighbor-to-neighbor exchanges of 
information about job opportunities or employment services. 

The program was delivered by local collaboratives comprising — at a minimum — the 
public housing authority, resident representatives, the welfare department, and the workforce 
development system. MDRC provided extensive technical assistance to facilitate program im- 
plementation in the six cities that were chosen through a national competition to be Job-Plus 
study sites: Baltimore, Maryland; Chattanooga, Tennessee; Dayton, Ohio; Los Angeles, Cali- 
fornia; St. Paul, Minnesota; and Seattle, Washington. 
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How Was Jobs-Plus Evaluated? 



Random Assignment of Housing Developments 

At each study site, one housing development was randomly selected (through a type of 
lottery) to operate Jobs-Plus from a matched pair or triplet of eligible public housing develop- 
ments nominated by the local public housing authority. The other one or two developments 
were assigned to a comparison group. Each development nominated had at least 250 units oc- 
cupied by families with a working-age adult. No more than 30 percent of these families could 
have an employed member, and at least 40 percent had to be receiving welfare. The random 
assignment to the program and comparison groups greatly reduced the risk of bias in the selec- 
tion of housing developments to participate in Jobs-Plus. Surprisingly good matches between 
the program developments and their comparison developments were obtained for all sites com- 
bined as well as within each site. 

Long-Term Trend Data on Residents 

Data on residents’ work and welfare receipt in both the Jobs-Plus and the comparison 
developments were used to examine the program’s impacts on residents and on their housing 
developments. These data were obtained from administrative records of government agencies 
for up to six years before and six years after Jobs-Plus was launched in 1998. In addition, two 
resident surveys were conducted (in 1998 and 2003) in three sites to assess whether the general 
quality of life within the housing developments had changed in terms of economic and material 
well-being, social conditions, personal safety, residential satisfaction, and child well-being. 



How Well Was Jobs-Plus Implemented — and in What Context? 

• Before the Jobs-Plus initiative was launched, living conditions were dif- 
ficult in both the Jobs-Plus and the comparison developments. 

Many public housing residents in the study sites faced challenges to employment, in- 
cluding limited education, lack of adequate child care, health or medical problems, and worry 
about crime and safety. Nevertheless, while residents expressed concern about problems in their 
housing development, three-quarters of them rated their development as at least a “good” place 
to live. Across the sites, there was considerable demographic diversity — for example, while 
three sites were predominantly African-American, the others had a more varied ethnic and ra- 
cial mix (including Southeast Asian and East African immigrants); in one site, as many as 22 
different languages were spoken. 
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• The Jobs-Plus model was ambitious and took over two years to imple- 
ment. Four of the six sites overcame numerous obstacles and operated 
programs of reasonable quality. 

Jobs-Plus’ s multicomponent approach — which was aimed at all working-age, nondis- 
abled residents and which included new rent rules to help make low-wage work “pay” — made 
the program one of the most comprehensive efforts ever attempted to improve work and welfare 
outcomes for public housing residents. The challenges of implementation were magnified by 
focusing the intervention on a diverse set of high-poverty housing developments within high- 
poverty communities and involving multiple public agencies and residents. To meet these chal- 
lenges, the sites had to enlist the active support of senior officials in the public housing authori- 
ties and other agencies; overcome cumbersome personnel and procurement policies of local 
housing authorities; create a new culture of collaboration among housing authorities, residents, 
and welfare, workforce, and social service agencies; deal with enduring issues of safety, sub- 
stance abuse, and other family crises; and, in some sites, adapt the services to suit a mix of im- 
migrant and native-born residents or to respond to high move-out rates. In addition, the pro- 
grams encountered some skepticism among residents, a situation that was not helped when, due 
to federal funding problems, the centerpiece of the program — the rent-based financial work 
incentives — was delayed for nearly two years. Furthermore, in one site (Seattle), Jobs-Plus 
also had to contend with a federal HOPE VI “tear down and rebuild” renovation initiative that 
displaced many of the residents soon after Jobs-Plus was fully in place. 

Despite many false starts and wrong turns — which took several years and considerable 
effort to overcome — four of the six sites (Dayton, Los Angeles, St. Paul, and Seattle) were 
able to build coherent programs of reasonable quality, making the demonstration a “fair test” of 
the Jobs-Plus model. Although the program was voluntary, many residents chose to make use of 
its services and rent-based work incentives, and, by a number of measures, Jobs-Plus informally 
reached many others in the developments as well. 

• The implementation of Jobs-Plus came at a time of momentous changes 
in welfare, workforce, and housing policy — and in the national econ- 
omy. These changes expanded work opportunities for public housing 
residents even in the absence of Jobs-Plus. 

As a consequence of the booming national economy and, perhaps, of the broad range of 
federal policy reforms, employment and earnings rose dramatically for residents both of the 
Jobs-Plus developments and of the comparison developments during the baseline period before 
the start of Jobs-Plus. By the time the program was launched in 1998, employment rates were 
higher than had been anticipated, and welfare receipt rates were lower, so that the margin for 
Jobs-Plus to “make a difference” was smaller than originally envisioned. Nonetheless, Jobs-Plus 
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still had considerable room to improve these outcomes further. In addition, rent-based work in- 
centives for all public housing residents were increasing due to reforms in federal housing law, 
and some work-related services were expanding. Nevertheless, the concerted effort by Jobs-Plus 
to provide additional services and incentives to residents of the program’ s developments beyond 
those available to residents of the comparison developments was successful. 



Did Jobs-Plus Make a Difference? 

MDRC looked at the impacts of Jobs-Plus from two perspectives: (1) the levels of work 
and welfare receipt among individuals (whether or not they continued living in their develop- 
ments) and (2) the levels of work and welfare receipt in the developments overall. It also exam- 
ined, in a more exploratory way, changes in a variety of community outcomes within the devel- 
opments. The findings described below represent effects for residents and developments that 
had access to Jobs-Plus relative to outcomes for residents and developments that did not have 
access to the program. 

Work and Welfare Impacts on Individuals 

• Across all six sites combined, once Jobs-Plus was in place, the program 
increased residents’ average annual earnings by 6.2 percent beyond 
what they would have been without the program. 

Panel A of Figure ES.l illustrates, for all six sites combined, the earnings of residents 
living in the Jobs-Plus or comparison developments in 1998 (a group of residents that is referred 
to as the “1998 cohort”). During the baseline period before the program began (from 1994 to 
1997), earnings for both groups were extremely similar and rose rapidly in response to the 
booming economy and changes in federal policies. 1 Their earnings remained similar during 
most of the program rollout period (1998 and 1999). Subsequently, from 2000 through 2003, 
when the program was most fully implemented, earnings for the Jobs-Plus group pulled well 
ahead of those for the comparison group. This difference illustrates the effect of Jobs-Plus on 
earnings for the 1998 cohort, which includes all targeted residents, whether they stayed in their 
original development or moved away. Based on these data, the estimated impact of Job-Plus on 
residents’ earnings averaged $498 per person per year over the four years after the rollout pe- 
riod. This means that, across all sites, the residents of the Jobs-Plus developments earned an 
average of $498 more per year than they would have earned in the absence of the program. 



'The baseline period began in 1992 for four sites and in 1994 for two sites, St. Paul and Chattanooga, be- 
cause earlier data were not available. 
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The Jobs-Plus Demonstration 



Figure ES.l 



Pooled Average Quarterly Earnings for the 
Jobs-Plus Group and Its Comparison Group (1998 Cohort) 

A. All Sites Combined 
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SOURCES: MDRC calculations using data from housing authority tenant (50058) records and state 
Unemployment Insurance (UI) wage records. 

NOTE: The 1998 cohort includes all residents of a Jobs-Plus development or a comparison 
development in October 1998 who were between 21 and 61 years old and were not listed as disabled 
by their public housing authority. 



ES-6 




• In three sites that built programs of reasonable quality, Jobs-Plus had 
an earnings effect that averaged 14 percent per year — and nearly 20 
percent in the fourth year. 

The all-site averages hide important findings at the site level. Panel B of Figure ES.l 
further illustrates the impacts of Jobs-Plus on residents’ earnings in the three stronger imple- 
mentation sites (Dayton, Los Angeles, and St. Paul) combined. There the annual impacts aver- 
aged $1,141 once the full program was in place. Note that the impacts in these sites grew larger 
over time. By the last year of follow-up, they had reached $1,540, which is nearly 20 percent 
higher they otherwise would have been. These impacts, which totaled $4,563 over four years, 
are especially noteworthy both because they persisted even after the onset of a national eco- 
nomic recession and because they represent “value added” by the program over and above any 
effects produced by concurrent reforms in the welfare, workforce, and public housing systems. 

The earnings impacts for the fourth strong site, Seattle, were also growing larger over 
time, but they then disappeared when its residents were relocated by a federal HOPE VI renova- 
tion initiative. In Baltimore and Chattanooga — sites that did not fully implement the program 
— Jobs-Plus had no earnings impacts. 

• Where it was implemented well, Jobs-Plus ’s effects on employment rates 
were also positive, but they were smaller, less consistent, and less fre- 
quently statistically significant. 

For the three stronger implementation sites combined, Jobs-Plus increased the average 
percentage of residents employed per quarter from 2000 through 2003 by an estimated 4.6 per- 
centage points (or by over 9 percent relative to what this rate would have been without the pro- 
gram). Because this finding is not statistically significant, there is considerable uncertainty 
about it. Nevertheless, because large and statistically significant effects on employment rates 
were observed for key subgroups at these sites, it seems reasonable to infer that the overall em- 
ployment impacts are real. This suggests that roughly two-thirds of the program’s effects on 
earnings at the sites were due to an increase in the number of persons employed. The remaining 
one-third was due to an increase in the amount earned per person employed, which represents a 
mix of increased employment stability, hours worked per week, and hourly wage rates. How- 
ever, there was no single simple relationship between the program’s estimated effects on em- 
ployment rates and earnings. For some subgroups, the findings suggest that almost all of the 
earnings gains produced by Jobs-Plus were due to an increase in the number of persons em- 
ployed. For other subgroups, the findings suggest that almost all of the program-induced earn- 
ings gains were due to an increase in the amount earned per person employed. 

• Where it was implemented well, Jobs-Plus was effective for many differ- 
ent types of public housing residents. 
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In the stronger implementation sites, Jobs-Plus had large positive earnings effects for 
many subgroups of residents. For example, it caused earnings to increase for men as well as for 
women, for residents who were receiving welfare when the program began and for those who 
were not, and for residents from different racial and ethnic groups. It also worked for subgroups 
of residents defined in terms of age, past employment, past duration of residence, and future 
residential mobility. 

• In the two sites with sizable populations of men, Jobs-Plus’s earnings 
impacts were exceptionally large for immigrant men, 

Jobs-Plus increased the average annual earnings of Hispanic men in Los Angeles by 
$3,248 (or 28 percent) and of Southeast Asian men in St. Paul by $2,129 (or 21 percent). Almost 
all these men were immigrants and members of two-parent families. The impacts are illustrated by 
the graphs in Figure ES.2. Those show that, during the baseline period, the earnings of the Jobs- 
Plus group and comparison group within each of these two subgroups were quite similar and that, 
after Jobs-Plus was fully implemented, the earnings of the Jobs-Plus group greatly surpassed those 
of the comparison group. Moreover, the effects continued to grow, and, by the final year of fol- 
low-up (2003), they reached $3,828 (a 35 percent gain) in Los Angeles and $3,366 (a 32 percent 
gain) in St. Paul. Over four years, these impacts totaled $12,994 and $8,5 17, respectively. 

• Although Jobs-Plus was effective in boosting earnings both for welfare re- 
cipients and nonrecipients, it was much more effective for nonrecipients. 

In the three stronger implementation sites, Jobs-Plus increased the average annual earn- 
ings of welfare recipients during each of the last four years of the study period by $761 (or almost 
11 percent) and those of nonrecipients by $1,654 (or 18 percent). This difference may reflect the 
fact that welfare recipients in comparison developments experienced a “push” toward work and 
had access to services and financial incentives through existing mandatory welfare-to-work pro- 
grams, time limits on benefits, and other features of welfare reform in their localities. In contrast, 
nonrecipients would not have been affected by these policies (although they were free to seek any 
other services they wanted). Thus, for the nonrecipients, Jobs-Plus may have represented a bigger 
additional (or net) intervention in their lives than it did for recipients. 

• Welfare receipt by residents dropped precipitously after Jobs-Plus was 
launched, but this decline was not related to Jobs-Plus. 

Instead, the drop in residents’ reliance on welfare was more likely due to forces such as 
a booming economy, welfare reform, and increases in the generosity of the Earned Income Tax 
Credit. These factors are viewed by many as causing the dramatic declines in welfare rolls that 
occurred throughout the United States at the time. 
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The Jobs-Plus Demonstration 



Figure ES.2 



Average Quarterly Earnings for the Jobs-Plus Group and Its Comparison 
Group, for Men in the Largest Demographic Subgroups in the Stronger 
Implementation Sites (1998 Cohort) 



A. Los Angeles - Hispanic Men 
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◄ Baseline Follow-Up ► 



B. St. Paul - Southeast Asian Men 
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◄ Baseline Follow-Up ► 



SOURCES: MDRC calculations using data from housing authority tenant (50058) records and state 
Unemployment Insurance (UI) wage records. 

NOTE: The 1998 cohort includes all residents of a Jobs-Plus development or a comparison 
development in October 1998 who were between 21 and 61 years old and were not listed as disabled 
by their public housing authority. 
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Work and Welfare Impacts on Public Housing Developments 

This study also sought to determine whether, because of Jobs-Plus, housing authorities 
would see an increase in the overall levels of employment and earnings and a reduction in welfare 
receipt among residents living in the Jobs-Plus housing developments at any given time — recog- 
nizing that people move in and out of public housing developments. For example, would the peo- 
ple who were helped by Jobs-Plus quickly move away, leaving no overall improvement in earn- 
ings or employment rates within the developments themselves? The findings suggest that: 

• Not surprisingly, when no impacts were produced on the sample of resi- 
dents who were followed over time (that is, the 1998 cohort, some of 
whom moved away), no year-by-year changes in outcome levels were 
observed within the developments. 

• When positive impacts were produced for the sample of residents fol- 
lowed over time, they yielded improvements in outcomes within the de- 
velopments — but by an amount that was inversely related to residents’ 
mobility. In other words, the more stable the resident population was, 
the greater the degree to which individual-level impacts were reflected 
by development-level impacts. 

Therefore, earnings gains for developments were largest in the two stronger implementation 
sites (Los Angeles and St. Paul) where move-out rates were lowest. 

Changes in Other Community Indicators 

• There is no evidence that the impact of Jobs-Plus on earnings for public 
housing developments (which were sometimes large but not transforma- 
tive) produced spillover effects on other community outcomes. 

Drawing on resident surveys in three of the six sites, the study found, in a largely de- 
scriptive analysis of community change, no indication that Jobs-Plus improved prevailing social 
conditions and the quality of life within the housing developments themselves, including eco- 
nomic and material well-being, personal safety, residential satisfaction, and child well-being. 
(The study did not measure quality-of-life changes among residents who moved away.) 



What Are the Implications for Public Policy? 

The Jobs-Plus demonstration shows that an employment-focused intervention that is 
based in public housing developments can work. Although Jobs-Plus proved challenging to im- 
plement, it eventually succeeded at four of the six study sites in increasing public housing resi- 
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dents’ earnings relative to what they would otherwise have been. Hence, this initiative offers one 
promising approach for helping to achieve the self-sufficiency objectives espoused by QHWRA, 
the 1998 federal housing reform law. 

Among the most striking findings from this study are that, even in some of the nation’ s 
poorest housing developments in tough urban environments, residents were more attached to 
the formal labor force than had been expected and that many more of them than were antici- 
pated responded to the expanding employment opportunities driven by the booming national 
economy of the 1990s. 

This study suggests important implications for policymakers to consider, including: 

• Jobs-Plus ’s effects compare favorably with those of other successful em- 
ployment interventions. When properly implemented, Jobs-Plus created 
earnings impacts at the high end of those found among many employment in- 
terventions that have been tested rigorously. This is particularly encouraging, 
given the limited evidence of effectiveness of self-sufficiency interventions 
in the housing and community development fields. 

• The success of Jobs-Plus in increasing the earnings of public housing 
residents was not limited to a narrow segment of the resident popula- 
tion; it was widespread across many subgroups of this population. When 
implemented properly, programs like Jobs-Plus can be effective for men as 
well as women and for native-born residents as well as for immigrants from 
very different parts of the world. 

• Improving Jobs-Plus’s impacts on employment rates might require ad- 
ditional efforts. The substantial pre-program rise in residents’ employment 
levels left Jobs-Plus with less room than anticipated to produce dramatic fur- 
ther improvement. Notwithstanding the positive employment impacts the 
program did have, a minority of residents remained largely out of the formal 
labor market. Thus, any effort to replicate the program should consider ways 
of reaching such harder-to-employ residents. 

• The rent breaks offered by Jobs-Plus encouraged residents to participate 
in the program and helped them increase their work efforts and earnings. 

(Normally, public housing residents’ rent is raised as their earnings increase; in 
Jobs-Plus, rent was held stable or rose less quickly than usual.) This suggests 
that, at a minimum, the more modest rent incentives that currently exist under 
the 1998 federal housing law should be fully implemented and aggressively 
marketed to residents, and perhaps expanded. The idea of combining rent- 
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based financial work incentives with services focused on work might also be 
worth rigorously testing in other housing assistance programs. 

• Jobs-Plus can aid the cause of welfare reform by improving the earnings 
of residents who are welfare recipients — even though the program had 
much larger effects on the earnings of nonrecipients. 

• Jobs-Plus has the potential to work in a variety of settings, given the 
demographic and geographic diversity of the sites in which it was success- 
fully implemented. Implementing the program was a difficult task that re- 
quired sustained attention and concentrated resources over an extended pe- 
riod of time. However, as further experience is gained with such initiatives 
and more is learned about implementing them, this process should become 
less difficult and time-consuming. 

• Strong housing authority leadership is vital. Successful replication of 
Jobs-Plus would require the sustained commitment of local public housing 
authorities to lead local collaboratives, to ensure that housing managers co- 
operate with the day-to-day operation of the program, to hold the program 
managers accountable for high performance, and to involve resident repre- 
sentatives in planning and operations. 

• Resident mobility matters. The Jobs-Plus findings caution that resident 
move-out rates greatly influence how earnings effects for individuals can 
translate into development-level effects. Thus, high rates of resident mobility 
would make the goal of substantially improving the income mix within pub- 
lic housing developments difficult to achieve through programs like Jobs- 
Plus alone. 

Finally, Jobs-Plus offers many practical lessons for constructing and operating labor 
market interventions, even outside public housing. In particular, the experiences of the six study 
sites speak directly to the challenges and opportunities of using “places” as the platform for a 
work-promoting intervention. They also point to many productive strategies for building part- 
nerships among multiple local agencies to address the employment needs of low-income popu- 
lations and for involving local residents in that process. 



ES-12 




Chapter 1 

The Origin and Goals of Jobs-Plus 



Can a multicomponent employment initiative located in public housing developments 
help residents get and keep jobs, earn more money, and improve the quality of their lives? To 
find out, a consortium of public and private funders led by the U.S. Department of Housing and 
Urban Development (HUD) and the Rockefeller Foundation sponsored the Jobs-Plus Commu- 
nity Revitalization Initiative for Public Housing Families (“Jobs-Plus” for short). 1 Designed to 
help policymakers learn “what works,” this research demonstration project has been testing an 
innovative employment strategy operating in public housing developments in six cities (or 
demonstration “sites”) across the United States. 

The designers of Jobs-Plus envisioned the program as a way to offer an economic boost 
to some of the nation’s poorest people and neighborhoods. Its principal goals were threefold: (1) 
to help people living in these very poor places increase their levels of employment and earnings; 
(2) by doing so, to help foster the emergence of a broader mix of incomes within those places; 
and (3) to achieve improvements in residents’ quality of life as a result of the gains in employ- 
ment and earnings. These goals are similar to several core objectives of the Quality Housing and 
Work Responsibility Act (QHWRA), the federal legislation passed by Congress in 1998 that 
also seeks to promote residents’ self-sufficiency through new economic incentives and opportu- 
nities to facilitate mixed-income communities and poverty deconcentration within public hous- 
ing. 2 Thus, the results of Jobs-Plus speak to the challenges, feasibility, and effectiveness of one 
approach for achieving these important national purposes. 

Jobs-Plus also has relevance beyond public housing. For example, many of the types of 
people it targets make up part of the low-income populations that welfare-to-work, workforce 
development, and community development organizations aim to help. Consequently, Jobs- 
Plus’ s mission of improving life outcomes for public housing residents partially overlaps with 
the mission of these other institutions, and, if it “works,” it will mean that a program based in 
public housing can help such institutions accomplish their broader purposes. In addition, the 



'in addition to the Rockefeller Foundation and HUD, the demonstration was funded by the U.S. Depart- 
ments of Health and Human Services and Labor; the Joyce, James Irvine, Surdna, Northwest Area, Annie E. 
Casey, Stuart, and Washington Mutual Foundations; and BP. The basic Jobs-Plus model was designed 
jointly by HUD, the Rockefeller Foundation, and MDRC. MDRC has been responsible for overall 
management of the demonstration, providing or arranging for extensive technical assistance to each 
participating city on the design and operation of its particular local approach, and for conducting the 
evaluation. 

“Said and Bogdon, 2003; U.S. Department of Housing and Urban Development, 1999. 
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particular operating strategies tried under Jobs-Plus — such as efforts to build new interagency 
and resident partnerships, promote rent-based financial incentives and other work supports, link 
work supports to employment services, involve resident volunteers in program outreach strate- 
gies, and take advantage of “place” in assisting residents with their employment needs — can 
inform efforts to try similar kinds of approaches in programs aimed at other low-income fami- 
lies living outside public housing. 

This report is the final one in a series produced as part of MDRC’s comprehensive 
evaluation of Jobs-Plus, and it focuses primarily on the program’s effectiveness, or “impacts.” To 
provide context and background for that analysis, the first few chapters discuss the demonstra- 
tion’s origins and goals, the characteristics and circumstances of the people that Jobs-Plus aimed 
to help, the local context within which the program had to operate, and the sites’ experiences in 
implementing the Jobs-Plus model. (Earlier publications, listed at the end of the document, ex- 
plore these topics in much greater depth.) The report then turns to the impact findings themselves, 
and it concludes with a discussion of the policy lessons and implications of those findings. 



Concentrated Poverty and Public Housing 

The problems plaguing inner-city communities can be particularly acute in many of the 
nation’s public housing developments, which rank among the most economically deprived 
neighborhoods in the country and are often part of larger neighborhoods with high rates of job- 
lessness and poverty. When the Jobs-Plus demonstration was launched in the late 1990s, almost 
54 percent of the nation’s 1.2 million units of public housing were located in high-poverty census 
tracts, and 68 percent were located in census tracts where 40 percent or more of working-age men 
had no regular employment. 3 Moreover, welfare recipients living in public housing were heavily 
concentrated in developments located in high-poverty neighborhoods. 4 This concentration of pub- 
lic housing in poor communities is believed by many to contribute to the social and economic 
problems experienced by residents living in those developments. It has also contributed to the ex- 
treme social isolation of many poor black families. 5 In addition, inner-city public housing devel- 
opments are widely believed to adversely affect the neighborhoods that surround them. 

By the 1990s, the population living in public housing had become substantially poorer 
than in recent decades. This trend emerged out of the changing mission of public housing itself. 
Since its inception during the Great Depression, this strand of the nation’s social safety net 
evolved from offering transitional shelter for unemployed and low-wage workers to providing 
permanent housing for the chronically nonemployed and impoverished. Families with working 

3 Newman and Schnare, 1997. 

4 Khadduri, Shroder, and Steffen, 2003. 

5 Massey and Denton, 1993. 
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members made up a minority of residents, especially in large inner-city housing developments. 
Nationally, in 1999, only 43 percent of public housing families with children relied on wages as 
their primary source of income, whereas public assistance — including cash aid under the Tem- 
porary Assistance for Needy Families (TANF) program, state-provided General Assistance 
(GA), and Supplemental Security Income (SSI) — was the primary source of income for 38 
percent of those families. 6 

In some cities, public housing residents appeared to be among the hardest people to 
employ among welfare recipients and other low-income groups, in part because of their poor 
education and job skills, meager work-relevant credentials, and array of personal problems or 
situations that make it difficult to work. 7 However, these personal constraints are not all that 
matter. Traditional public housing rent rules, under which rent usually increases as earnings rise, 
have long been thought to discourage many residents from working because they see little to 
gain. Furthermore, the mere circumstance of living in public housing may also impede work. 
This might happen because of the stigma that public housing casts on its residents in the eyes of 
many employers in the community, the physical or social separation of its residents from parts 
of the city or region where jobs are more abundant, and the influence of a social environment 
within the housing developments that discourage work. 

In response to these trends, policy experts began to consider new responses. Attention 
focused on a few very different kinds of strategies, including changing who moves into public 
housing, helping residents move out with portable rental assistance, and building the employ- 
ment capacity of existing residents. 8 

• Changing who moves in. One way to transform public housing develop- 
ments into places with a better mix of incomes is for housing authorities to 
recruit more working families. Indeed, QHWRA, the 1998 federal housing 
legislation, encourages and makes it easier for them to do just that by giving 

6 U.S. Department of Housing and Urban Development, 2000. General Assistance is cash and/or in-kind 
support that some states and localities provide to eligible persons who do not qualify for federal cash assistance 
(such as single adults and childless couples). Supplemental Security Income is a federal program for low- 
income disabled adults. More recent HUD data (U.S. Department of Housing and Urban Development, 2004) 
show that about 30 percent of all households (including those headed by elderly and disabled people) had any 
wages. Khadduri, Shroder, and Steffen (2003, Table 2.1) estimate that about 13 percent of all public housing 
units were occupied by families receiving TANF in 2000, which — following the steep national decline in the 
welfare rolls over the course of the 1990s — is down substantially from 23 percent of units occupied by fami- 
lies receiving Aid to Families with Dependent Children ( AFDC) in 1996. Among just the 550,000 public hous- 
ing units occupied by families with children, about 29 percent were receiving TANF in 2000, down from the 
51 percent who were receiving AFDC in 1996. For further information and analysis concerning the relation- 
ship between welfare reform and housing assistance, see Newman (1999) and Sard and Bogdon (2003). 

7 Riccio and Orenstein, 2003. 

8 For a broader review of housing assistance and affordable housing strategies, see Katz and Turner (2004). 
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them more discretion to set admissions preferences. 9 However, this approach 
would do nothing directly to improve the employment prospects of residents 
already living in public housing or of new residents who do not work. More- 
over, convincing larger numbers of working families to move into the devel- 
opments may be a formidable challenge, given the negative reputation of 
public housing in many cities. 

• Helping residents move out with portable rental assistance. One ap- 
proach that might help people already living in public housing is to offer 
them portable rental assistance (such as rent vouchers) that subsidizes the 
rent they pay to private landlords willing to lease to voucher recipients. 10 
With such assistance, they might be able to relocate to areas where job op- 
portunities are better and where they might connect better to informal social 
networks through which information about job opportunities flows. In one 
innovative test of this concept sponsored by HUD, the Moving To Opportu- 
nity (MTO) demonstration is assessing the effects of offering public housing 
residents either regular Section 8 rent vouchers or special Section 8 vouchers 
that could only be used in low-poverty neighborhoods (that is, where less 
than 10 percent of the population were poor). The offer usually came with 
some relocation assistance but without any special employment assistance. 
Interim results from the evaluation of MTO show that neither type of 
voucher led to increases in residents’ employment or earnings — at least 
through two years of follow-up — although the vouchers did generate some 
positive impacts on a mix of noneconomic outcomes. 11 

• Building the employment capacity of existing residents. An alternative to 
helping public housing residents move to more promising communities or to 
recruiting more working people as tenants is to try to increase employment 
and earnings among current tenants. Jobs-Plus falls into this class of inter- 
ventions. Self-sufficiency initiatives attempting to do this are certainly not 



9 Sard and Bodgon, 2003; U.S. Department of Housing and Urban Development, 1999. 

"’Under tenant-based Section 8 rules, voucher holders pay at least 30 percent of their adjusted income in rent 
to a qualifying landlord, and HUD makes up the difference up to a standard set for modest housing in the area. 

1 'Oix et al., 2003. MTO and Jobs-Plus were two of three demonstration research projects that HUD (with 
private foundations) launched to test innovative strategies for reducing concentrated poverty and some of its 
deleterious effects. A third one, called “Bridges to Work,” emphasized reverse commuting to connect inner- 
city residents (but not necessarily public housing residents) to jobs in the suburbs. See Reardon, 2001. 
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new to public housing, but Jobs-Plus is considerably more ambitious in scope 
and intensity. 12 

Increasing the amount of earnings within the tenant population — by building their 
earnings capacity or by getting more working people to move in — may be critical for ensuring 
the future viability of public housing itself as a source of decent, affordable housing for low- 
income families. When Jobs-Plus was first conceived in the 1990s, it was widely feared that 
welfare time limits and other restrictions on access to welfare benefits might leave many public 
housing residents less able to contribute to their rent, requiring greater subsidies to cover the 
cost of housing them. 13 In addition, reductions in federal operating subsidies to local public 
housing authorities could make it harder for those authorities to fill the gap left by declining rent 
revenues. Thus, increasing residents’ earnings — out of which they could pay more rent — 
could help make up some of this difference. 

Although Jobs-Plus represented one way to address employment problems in public 
housing, it was also conceived as a kind of community intervention; the community, in this 
case, was the public housing development itself. Over the past two decades, community initia- 
tives have become an increasingly popular approach for fighting geographically concentrated 
joblessness and poverty. 14 Like all such initiatives, Jobs-Plus targeted a definable geographical 
area to be the focus of its efforts to help low-income people improve the quality of their lives. It 
sought to help individuals, whether or not they choose to continue living in public housing, as 
well as the places themselves. However, rather than attempting to achieve a variety of commu- 
nity change goals simultaneously (a common strategy of comprehensive community initiatives), 
Jobs-Plus focused on a single goal: improving employment-related outcomes. This was the 
driving force around which all program elements were to be organized. It was hypothesized 
(drawing on the work of William Julius Wilson and others) 15 that, by substantially increasing 
residents’ rates and stability of employment, other improvements in residents’ quality of life 

l2 See Bogdon, 1999. Other current efforts include HUD's Step-Up program, which provides apprentice- 
ship training in home construction trades with on-the-job experience offered through rehabilitation projects 
within public housing developments. Another is the Family Self-Sufficiency program, in which housing au- 
thorities offer employment and training referrals, job counseling and support, and opportunities to have rent 
increases that result from higher earnings put into a special escrow savings account for residents. Innovative 
strategies attempting to enhance residents’ self-sufficiency are also being tried under HUD’s Moving to Work 
demonstration, which, in a test of devolution, frees selected housing authorities from a variety of restrictions on 
their decision-making authority imposed by the 1937 federal housing law. For an overview and assessment of 
this demonstration, see Abravavel et al. (2004). It should be noted that, of the 30 site slots that were initially 
included in the Moving to Work demonstration, HUD set aside six for inclusion in the Jobs-Plus demonstra- 
tion. For an examination of a long-standing self-sufficiency and home ownership program operated by the 
Charlotte Housing Authority in North Carolina, see Kleit and Rohe (2005). 

13 See, for example, Naparstek, Dooley, and Smith, 1997. 

14 Aspen Institute, 1997. 

15 See, for example, Wilson, 1996; Dickens, 1999; and Jargowsky, 1997. 



5 




would follow — including, for example, reduced poverty and material hardship, crime, sub- 
stance abuse, and social isolation; increased general satisfaction with living in the community; 
and improved outcomes for children. Although the evaluation would not be able to test whether 
Jobs-Plus had all these effects, the program design was premised on the notion that improving 
residents’ employment and earnings could be an engine for broader social changes for them per- 
sonally and for life within their public housing developments. 



The Jobs-Plus Model 

With the lessons from past employment initiatives in mind, the demonstration’ s main de- 
signers — HUD, the Rockefeller Foundation, and MDRC — believed that Jobs-Plus should ad- 
dress, in depth, a combination of problems widely believed to conspire against sustained employ- 
ment among public housing residents. These included poor preparation for work; inadequate 
knowledge about seeking work; personal, family, or situational problems (such as lack of child 
care or transportation) that can impede work; absence of a strong financial incentive to take a low- 
wage job; and living in a social environment that does not encourage or facilitate work. 

A Broadly Targeted, Three-Component Intervention 

The designers thus conceived of a broad, three-component intervention (see Table 
1.1). 16 One component would focus on employment-related, sendees and activities. These would 
draw on the best practices of past employment initiatives and include such activities as instruc- 
tion in job search skills, combined with some education and training. Also necessary would be 
assistance with child care and transportation, to make it feasible to work and participate in 
work-related activities. Some of these services could be offered on-site at the developments, but 
the great diversity in residents’ job readiness and service needs would also require access to 
networks of existing services in the local community. The second component would involve the 
creation of new financial incentives to work. These would include new public housing rent rules 
to replace the traditional HUD rules under which residents paid 30 percent of their income (after 
certain adjustments) in rent, imposing an implicit “tax” on their earnings. The new rules would 
help make low-wage work “pay,” by reducing the extent to which gains in earnings are offset 
by increases in rent. The program’s third component, community support for work, would in- 
volve strengthening residents’ work-supporting social capital. For example, Jobs-Plus would 
attempt to foster work-related information-sharing, peer support, and mutual aid among resi- 
dents and with people living outside public housing. 



16 For further details on the rationale behind these features of the program, see Riccio (1999). 
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The Jobs-Plus Demonstration 



Table 1.1 

The Jobs-Plus Model 



The Jobs-Plus model calls for “saturation” targeting of all working-age, nondisabled residents with: 

• Employment-related services and activities to help residents secure and retain employment, 
including job search instruction, education programs, vocational training, and support ser- 
vices, such as child care and transportation assistance. 

• Financial incentives to work, consisting of changes in public housing rent rules that help to 
“make work pay” by reducing the extent to which higher earnings from work result in in- 
creases in rent, which may discourage work. 

• Community support for work, which seeks to strengthen social ties and activities among 
residents to support their job preparation and work efforts — for instance, by fostering 
neighbor-to-neighbor exchanges of information about job opportunities or various employ- 
ment services available through Jobs-Plus. 



Jobs-Plus’ s multicomponent approach is fundamental to the program’s underlying theory 
— the vision of how it is expected to produce large impacts on employment and earnings. Ac- 
cording to this theory, tackling a variety of obstacles to work through a combination of employ- 
ment services, financial incentives, and social network strategies would enhance residents’ interest 
in and commitment to working, their capacity to look for and find work, their skills to qualify for 
better jobs, and their knowledge about job opportunities. These changes, in turn, would increase 
their participation and success in the labor market and reduce their reliance on welfare. Increases 
in employment and earnings, if they occurred, would then foster improvements in the quality of 
life within the developments and residents’ own personal and family well-being, growing out of 
residents’ increased income and more productive engagement in society. 

Jobs-Plus is also distinctive because of its attempt to implement all three program com- 
ponents at saturation levels within the participating public housing developments. That is, it was 
to be targeted toward all residents who were of working age and not disabled (as classified by 
the housing authority according to HUD rules). Thus, at the very least, all such residents were to 
be exposed to new work-promoting “messages” from program staff and neighbors. Further- 
more, the families who participated could benefit from the new financial incentives and take 
advantage of a diverse array of services and supports. 

The saturation focus of the intervention was seen as a way to promote and strengthen a 
normative environment focused on work throughout participating housing developments. It was 
also viewed as having the potential to spark escalating change within the developments: Target- 
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ing the intervention toward the entire working-age population of a public housing development 
may produce a critical mass of employed residents (reaching a “tipping point”) 17 whose experi- 
ences would generate momentum for change across the development. As these vanguard work- 
ers grew in number, their visibility and influence as role models would be enhanced. Their own 
success would signal to others the feasibility and benefits of working, elevate and strengthen the 
social norms that encourage work, foster the growth of work-supporting social networks, and, 
ultimately, contribute to still more residents getting and keeping jobs. 

A Collaborative Approach 

Jobs-Plus targeted public housing residents, but its national sponsors did not intend 
for it to be a program solely “owned” and operated by public housing authorities. Instead, 
they envisioned it as a locally collaborative undertaking and required that each participating 
city tap a reservoir of local knowledge, technical expertise, and resources to adapt the generic 
model to local circumstances and operate the program. The sponsors hoped that, by taking 
this approach, the program would stand a much greater chance of success than if any single 
local partner were to design and operate it alone or if it were to be designed in detail by the 
national demonstration partners. 

To qualify for the demonstration, each local collaborative had to include, at a minimum, 
the following four partners: the public housing authority, resident representatives, the welfare 
department, and the workforce development system (represented by the agency operating since 
1998 under the Workforce Investment Act, or WIA). Each of these partners could bring some- 
thing special to the task of designing and implementing an effective Jobs-Plus program but was 
limited in what it could do alone. For example, the housing authorities had access to HUD re- 
sources and controlled many policies affecting housing developments and their tenants, but they 
needed the experience and resources of the welfare department and the workforce development 
agency in providing employment and social services. At the same time, these agencies had little 
knowledge of the circumstances of public housing residents, who formed a sizable percentage 
of their caseloads. Resident representatives on the collaboratives could bring an in-depth aware- 
ness of their communities and service needs and could foster community trust and “buy-in” for 
the program. Finally, other local organizations were expected to join as a source of services, 
expertise, and other resources that would help advance Jobs-Plus’ s employment mission. 



17 Gladwell, 2001. 




Recruiting and Selecting Sites 

Cities were chosen for the demonstration through a national competition sponsored by 
HUD and the Rockefeller Foundation and managed by MDRC. 18 Together, these three organi- 
zations established the site selection criteria and picked the sites. They did not attempt to recruit 
cities and local housing authorities that, as a group, were nationally representative. This would 
not have been feasible. Instead, they placed priority on recruiting a diverse set of sites where 
joblessness in public housing was a serious problem and where there appeared to be a good op- 
portunity to build and test a large-scale, well-managed employment initiative. 

Only large housing developments — defined as having at least 250 family-occupied 
units, not counting those occupied only by people 62 years old or older — could qualify for the 
demonstration. Qualifying housing authorities had to have at least two such developments. In 
addition, according to housing authority records, no more than 30 percent of families living in 
these developments could have an employed member, and at least 40 percent had to be receiv- 
ing Aid to Families with Dependent Children (AFDC). These criteria were meant to ensure that 
Jobs-Plus would be tested in places where the need for an employment intervention was great 
and where the scale of the intervention could be substantial. 19 Across the continental United 
States, 442 housing developments managed by 53 local housing authorities met these criteria. 20 

The quality of local housing authority management was also important. Because Jobs- 
Plus was a new intervention with details of the design to be developed at the local level, even 
the most effective housing authorities would be challenged by it, and by the added demands of a 
research demonstration. Thus, an effort was made to screen out housing authorities that were 
having substantial difficulty managing basic housing services — although this was not always 
easy to assess. 

Furthermore, cities eligible for the demonstration had to be willing to adopt a collabora- 
tive strategy for designing and operating the intervention, and at least some of the key local 
partners had to have collaborated successfully in the past. The core role anticipated for the hous- 
ing authority and for the welfare and job training systems made their commitment essential. Cit- 
ies also had to show a willingness to include residents as partners, and existing resident organi- 
zations had to have a reasonable capacity to play that role. 



18 For additional details on the site selection process, see Riccio (1999). 

19 The sample-size needs of the demonstration’s evaluation design were another consideration. 

20 This estimate is based on MDRC calculations using 1993 data from HUD’s hiformation Services Division 
of Public and Indian Housing. According to 1998 data, approximately 3,300 public housing authorities nationwide 
manage a total of 13,919 housing developments. Approximately 10 percent of these properties (1,396) include 
200 or more units, accounting for 600,206 units in total (or 46 percent of all public housing units nationwide) 
(Council of Large Public Housing Authorities, 2004). 
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Finally, the local partners had to be willing and able to meet the demands of a rigorous 
and innovative research design. As described below, this design limited the housing authority’s 
role in choosing the development in which Jobs-Plus would be operated and required its coop- 
eration with substantial data collection efforts. 

In June 1996, an invitation to submit a statement of interest in the demonstration was 
sent to 50 of the 53 cities where, according to nationally available data, the public housing au- 
thority had the types of developments that were being sought. Attesting to the importance that 
housing authorities and other city agencies ascribed to the goals of the project, responses were 
received from 42 cities. By August 1996, 15 cities had been chosen to develop preliminary 
plans for a Jobs-Plus program, and, of these, seven were selected as the final set of sites in 
March 1997. Six of these cities continued into the main stage of the project: 21 Baltimore, Mary- 
land; Chattanooga, Tennessee; Dayton, Ohio; Los Angeles, California; 22 St. Paul, Minnesota; 
and Seattle, Washington. 

An intensive planning process continued in each of the selected sites for roughly a year. 
To help each collaborative develop its plans and then implement the Jobs-Plus components, 
MDRC deployed special “site representatives” and other experts to provide ongoing operations- 
related technical assistance. 

Over the course of the demonstration, important changes took effect in two sites that af- 
fected their role in the evaluation. In Seattle, the housing authority received a federal HOPE VI 
grant in 1999, which is being used to tear down and rebuild the Rainier Vista development where 
Jobs-Plus was located. 23 Because of the temporary dislocation of the residents that demolition and 



“'Cleveland, Ohio, had also been included among the seven cities selected to participate in the demonstra- 
tion. However, in 1999, due to a shift in local priorities, Cleveland left the demonstration by mutual agreement 
between its housing authority and the national Jobs-Plus sponsors and MDRC. 

22 Unless otherwise indicated, all references in this report to the Jobs-Plus development in Los Angeles re- 
fer to William Mead Homes. However, in Los Angeles, a second housing development — Imperial Courts — 
was selected to operate Jobs-Plus, in addition to William Mead Homes. For a variety of reasons, it was not 
possible to select a comparison development matched to Imperial Courts, in order to estimate the effectiveness 
of its Jobs-Plus program. Consequently, Imperial Courts is not included in the impact analysis and is excluded 
from this report, although it has been included in all of MDRC’s earlier implementation research on Jobs-Plus. 

“ HOPE VI is a HUD program that is aimed at redeveloping the most “severely distressed” housing 
projects across the country. The redevelopment process involves replacing public housing units with 
apartments or townhouses, some of which will become available at market rate to working families in an 
effort to reduce the concentration of poor households in the development communities. The local housing 
authority must use some HOPE VI funds to offer supportive services to residents who are relocated dur- 
ing the demolition, to help them find housing on the private market. However, housing authorities also 
have the option of offering — in addition to housing search assistance — various employment-related 
services to prepare residents for employment and life as private housing tenants (U.S. Department of 
Housing and Urban Development, 1999). For an assessment of the accomplishments of HOPE VI, see 
Popkin et al. (2004). 
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reconstruction has entailed — along with the infusion of special funds and requirements associ- 
ated with the HOPE VI initiative — the circumstances within which the Seattle program would 
continue to operate were unique among the Jobs-Plus sites. Consequently, Seattle was withdrawn 
from the national Jobs-Plus demonstration. However, because the site continued to operate a Jobs- 
Plus program at the development in modified form as the centerpiece of its HOPE VI community 
and supportive services plan (under the name “HOPE-Plus”), MDRC has continued to evaluate 
the Seattle program in a companion study, but with a scaled-back research design. 24 Where possi- 
ble, findings on Seattle’s program are included in sections of this report. 

In Chattanooga, the housing authority, MDRC, and the lead demonstration funders mu- 
tually agreed in April 2002 to change the Jobs-Plus program there into one that would offer only 
the financial incentives component, ending attempts to develop a full-scale program that also 
included employment-related services and community support for work. A number of factors 
prompted this decision, including the site’s lack of progress in operating the latter two compo- 
nents at a sufficient level of quality and the housing authority’s decision to bring its develop- 
ments under the management of a private contractor. The demands of implementing this priva- 
tization initiative would have limited the attention that the housing authority could give to a full 
Jobs-Plus program. The site’s official transition into a financial-incentives-only program was 
completed by the late summer of 2002, although the housing authority did continue to offer 
residents some on-site job search guidance and help accessing employment-related services. 25 



Evaluating Jobs-Plus 

As previously mentioned, Jobs-Plus is not the first employment intervention in public 
housing, but its scale and scope surpass those of other current and past initiatives. It is also the 
subject of the most in-depth evaluation of any other such program in public housing — or of 
any undertaken as part of a comprehensive community initiative. 26 

The saturation and place-based nature of Jobs-Plus made it impossible to assess the ef- 
fectiveness of the intervention using a traditional randomized experiment, a method widely 
viewed as the most credible way of determining a program’s effectiveness. In such an experi- 
ment, individuals are randomly assigned to either a program group, which receives the new in- 
tervention, or a control group, which does not. The estimate of the effect — or “impact” — of 



24 The companion evaluation was sponsored by HUD, the Stuart Foundation, and the Seattle Housing Au- 
thority. For a detailed account of the implementation of Jobs-Plus in conjunction with HOPE VI in Seattle, see 
Liebow et al. (2004). 

25 See Bowie, 2003. 

26 For a review of past studies, see Bodgon (1999). For a discussion of the many special difficulties of produc- 
ing credible evidence on the effectiveness of comprehensive community initiatives, see Hollister and Hill (1995). 
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the program is the difference between the groups on the outcomes of interest, such as average 
earnings. It was not possible to use this methodology for evaluating Jobs-Plus because the pro- 
gram’s services, incentives, and supports for work were to be targeted toward all working- age, 
nondisabled residents at each development. This made it impossible to create a randomly se- 
lected control group of individuals within a development who would be left “untouched” by the 
program. The evaluation therefore had to rely on an alternative approach. 

An Innovative Research Design 

The approach that was adopted, which is described in Chapter 4, involved randomly as- 
signing entire housing developments (rather than individual residents) to either a program group 
or a comparison group within each site. Coupled with this is a comparative interrupted time-series 
analysis of administrative records that compares each groups' earnings trend before and after the 
introduction of Jobs-Plus. For this design to work, the housing authority in each candidate city had 
to have at least two — and preferably three — developments that would qualify for Jobs-Plus and 
that housed demographically similar tenants. It also had to allow MDRC to determine randomly 
which one of these developments would be selected to operate the program. The other one or two 
developments would become part of a comparison group where research would be conducted but 
Jobs-Plus would not be operated. The purpose of the comparison group was to show how resi- 
dents’ employment, earnings, and welfare trends would have changed in the absence of Jobs-Plus. 
In other words, the change in outcomes for the comparison group would establish the benchmarks 
for assessing the “added value” of Jobs-Plus. As shown in Table 1.2, a total of 15 housing devel- 
opments were used for the final impact analysis: six that operated Jobs-Plus and nine that served 
as comparison developments. These program and comparison developments were generally sepa- 
rated by a distance of several miles, so they were not part of the same immediate community. 
These developments had roughly between 300 and 500 units each. 

The random allocation process was used in order to avoid systematically selecting for 
Jobs-Plus the best managed of the available developments or those that enjoyed the most favor- 
able conditions for achieving employment outcomes. In other words, it was an attempt to set up 
a “fair” comparison so that if better outcomes were observed for residents of the program de- 
velopments, these improvements could more confidently be attributed to Jobs-Plus rather than 
to other factors. Although the total number of developments is small, this process offers at least 
some assurance that, when all the sites are combined, those developments assigned to the pro- 
gram group are generally similar to those allocated to the comparison group. Of course, random 
assignment helps little in this regard within each site, since only two or three developments are 
available per site. However, the fact that, within each site, the developments were demographi- 
cally similar to begin with increased the likelihood that the types of residents living in the pro- 
gram and comparison developments would be comparable. As subsequent chapters demon- 
strate, the program and comparison developments within each site do include residents who, on 
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average, were extraordinarily well matched in terms of their prospects for success in the labor 
market in the absence of Jobs-Plus. This is evident from their long-term pre-program employ- 
ment and earnings trends as well as their demographic characteristics, and these conditions set 
the stage for conducting a credible impact analysis. 



The Jobs-Plus Demonstration 
Table 1.2 

Cities and Housing Developments in the 
Jobs-Plus Impact Study 



City 


Jobs-Plus Development 


Comparison Development(s) 


Baltimore 


Gilmor Homes 


Perkins Homes 
Somerset Courts 


Chattanooga 


Harriet Tubman Homes 


College Hill Courts 
Emma Wheeler Homes 


Dayton 


DeSoto Bass Courts 


Arlington Courts 
Parkside Homes 


Los Angeles 


William Mead Homes 


Dana Strand Village 


St. Paul 


Mt. Airy Homes 


Roosevelt Homes 


Seattle 


Rainier Vista Garden 
Community 


Yesler Terrace 



This strategy for testing the impacts of Jobs-Plus is unique among evaluations of public 
housing employment programs and comprehensive community initiatives. Although not as robust 
a research design as a social experiment involving the random assignment of individuals, it holds 
the potential to supply unusually reliable evidence of the impacts of a place-based intervention. 

Jobs-Plus’ s dual emphasis on “helping individuals” and “helping places” presented an- 
other analytical challenge. If residents who benefit from Jobs-Plus quickly move away, their 
contribution to average earnings or to the employment rate within a development at any given 
time will be less than if those residents do not move. Taking this into account, the impact study 
assesses Jobs-Plus’ s “success” from two perspectives: (1) What happens to individuals, whether 
or not they remain in public housing? and (2) What happens to the year-to-year employment 
and earnings levels in the developments themselves? The evaluation also explores whether any 
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improvements in development-level employment and earnings lead to improvements in tenants’ 
quality of life or conditions at the developments. 

Data for the impact analysis come from a variety of sources. The main sources include 
housing authority records that provide information on residents' background characteristics and 
their tenure in public housing, state Unemployment Insurance (UI) wage records, and welfare 
payment records (the latter was available for only three sites). These data were collected to cover a 
period extending back to 1992 and through the end of 2003. They make it possible to determine 
how similar residents’ employment, earnings, and welfare trends were prior to the start of Jobs- 
Plus and whether, afterward, residents of the Jobs-Plus developments proceeded along a more 
positive trajectory than their counterparts in the comparison developments. These trends were 
measured for all sample members, regardless of whether they moved out of or continued to live in 
public housing. 

In addition to these data, two surveys of residents were conducted: a baseline wave, 
fielded near the start of Jobs-Plus operations (1998-1999); and a follow-up wave (except in Seat- 
tle), fielded four to five years later, during the final year of the demonstration’ s operational pe- 
riod (2003). The surveys were limited to tenants of the developments at either of these points in 
time. They provide a basis for assessing whether, as hoped, residents of the Jobs-Plus develop- 
ment made greater use of work-promoting services and financial incentives than did their com- 
parison development counterparts. A more extensive version of the follow-up survey was con- 
ducted in three of the sites 27 and offers a basis for inferring whether Jobs-Plus contributed to 
important changes in social patterns within the developments and in residents’ quality of life. 
Other data were also collected on residents’ use of Jobs-Plus services and rent-based financial 
incentives in the program developments from housing authority and program records. 

Finally, the evaluation included an intensive effort to collect qualitative data. A combi- 
nation of on-site field researchers and central MDRC staff spent many hours over the life of the 
demonstration observing the operation of the collaboratives and the program, conducting open- 
ended interviews with residents, and investigating activities at the comparison developments. 

Using all these data, the evaluation assesses the overall feasibility and effectiveness of 
Jobs-Plus. The analysis is structured to an important degree around the pathways of change that 
were assumed in the design of the Jobs-Plus model. Figure 1.1 depicts these pathways. The 
analysis thus examines how well the features of the program model were implemented; whether 
Jobs-Plus increased residents’ efforts to prepare for and look for work; whether these efforts, in 
turn, improved their labor market and welfare outcomes; and whether any improvements in 
those outcomes yielded positive “spillover” effects on residents’ quality of life. 



27 The fuller version of the survey was not conducted in all sites because of cost considerations. 
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The Jobs-Plus Demonstration 



Figure 1.1 

Hypothesized Effects of Jobs-Plus 
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A Changing National Context 

Jobs-Plus did not operate in a policy vacuum, as can be seen in Figure 1.2. When the 
demonstration was launched in the latter part of the 1990s, the states and localities were imple- 
menting the major overhaul of the nation’s welfare system called for under the 1996 federal leg- 
islation, the Personal Responsibility and Work Opportunity Reconciliation Act (PRWORA). 
Under that legislation, states were required to impose time limits on receipt of federal welfare 
benefits and to expand requirements for recipients to take part in welfare-to-work programs. 
Many states also introduced more generous earnings disregards — that is, rules that allowed 
recipients to earn more money before losing their welfare grants — to help make low-wage 
work “pay.” Soon after the Jobs-Plus sites had been selected, the federal government replaced 
the Job Training Partnership Act (JTPA) with the Workforce Investment Act (WIA) of 1998, in 
the hope of providing better and easier access to job placement assistance, education, training, 
and other workforce services for low-income populations. And, in 1998, the federal government 
overhauled its housing policies with passage of the Quality Housing and Work Responsibility 
Act, which, as previously mentioned, included some provisions to encourage greater employ- 
ment among public housing residents. The 1990s also saw a large expansion of the federal 
Earned Income Tax Credit (EITC), which supplements the earnings of low-wage workers. 

As Jobs-Plus was rolling out, the national economy was in the midst of an economic 
boom that had brought unemployment rates to their lowest levels in decades. By 1996, the na- 
tional rate stood at 5.4 percent, and it continued to fall, to 4.0 percent in 2000. However, the pic- 
ture had changed by 2001, when the country entered a recession, 28 and the unemployment rate 
rose over the next few years, to 6.0 percent in 2003 — the last year in the operational phase of 
the demonstration. 

All these changes may have had some influence on the labor market decisions of resi- 
dents living in the Jobs-Plus developments as well as of residents in the comparison develop- 
ments. In and of themselves, the changes may have contributed to increased employment and 
earnings and to reduced reliance on welfare among public housing residents. Consequently, the 
Jobs-Plus impact analysis is a test of whether the program can make a difference in residents’ 
outcomes above and beyond what these residents might have accomplished in this changing 
context in the absence of Jobs-Plus. 



28 The National Bureau of Economic Research identified March 2001 as the date when the national reces- 
sion began (Hall et al., 2004). 



16 




The Jobs-Plus Demonstration 
Figure 1.2 

Time Line and National Context of the Jobs-Plus Demonstration 
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SOURCES: Public documents; MDRC field research; site documents; and (for unemployment data) U.S. Department of Labor, Bureau of Labor 
Statistics, Local Area Unemployment Statistics, http://data.bls.gov, data extracted on February 3, 2005. 

NOTES: TANF = Temporary Assistance for Needy Families created by the Personal Responsibility and Work Opportunity Reconciliation Act; WIA = 
Workforce Investment Act; QHWRA = Quality Housing and Work Responsibility Act. 








The Remainder of This Report 

Chapter 2 begins the comprehensive assessment of Jobs-Plus by describing the widely 
varying settings and the highly diverse types of public housing residents for whom the model 
was tested. Chapter 3 then analyzes the sites’ struggles and accomplishments as they worked to 
translate the core elements of the Jobs-Plus model into a functioning program suitable to their 
particular real-world circumstances. 

Chapters 4 and 5 assess the effectiveness of Jobs-Plus in improving residents’ employ- 
ment, earnings, and welfare receipt — first through a longitudinal analysis of individuals (some 
of whom moved out of public housing) and then from the perspective of public housing devel- 
opments themselves, which experienced a steady turnover in their tenant populations. 

Chapter 6 examines indicators of community change over the study’s five-year period, 
starting with the year that Jobs-Plus operations began and ending with the year that the opera- 
tional phase of the demonstration concluded. Using resident survey data, that analysis offers 
limited but important evidence about changes on a variety of dimensions of the quality of life 
among residents in the Jobs-Plus and comparison developments. 

Chapter 7 concludes the report with a discussion of the policy lessons and implications 
of the evaluation’ s findings. 
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Chapter 2 

The Residents and Their Communities 



Jobs-Plus was intended to improve the well-being of residents of some of the poorest 
communities in the country — public housing developments distinguished by low rates of em- 
ployment and high reliance on welfare. This chapter shows that, in the six developments selected 
to operate the program, residents faced many barriers to employment, and when they did work, 
the jobs they held were usually very low paying and often without fringe benefits. Limited educa- 
tion, lack of adequate child care, health or medical problems, and worry about crime and safety 
were common. Reliance on welfare was high; poverty was widespread; and living conditions 
were difficult. Residents expressed great concern over serious social problems in their develop- 
ments, and yet three-quarters rated their development as at least a “good” place to live. 

The racial and ethnic composition of the tenant populations differed across the six sites. 
For example, while three sites were predominantly African-American, the others had a more 
varied ethnic and racial mix. Moreover, the combination of large and midsize cities in which the 
developments are located confronted Jobs-Plus with varying labor markets, housing markets, 
and social conditions. These types of variation in local context make it possible to ask whether 
Jobs-Plus can “work” under a range of circumstances. 

Although most of this chapter describes the people and conditions at the Jobs-Plus de- 
velopments, the same descriptions also apply to the comparison developments. This high- 
quality match is important because it lends credibility to the evaluation’s impact findings dis- 
cussed in Chapters 4 and 5. 



Characteristics of the Surrounding Communities 

Census data from 2000 indicate that the areas in which the Jobs-Plus housing develop- 
ments are located are similar to those featured in the literature on high-poverty communities. 
They are primarily census tracts populated by people of color (see Appendix Table A.l). They 
are also tracts in which households headed by single parents are common and large numbers of 
adults do not have a high school diploma. Three of the six Jobs-Plus developments (in Los An- 
geles, St. Paul, and Seattle) are located in tracts where a high proportion of residents — from 26 
percent to 62 percent — were foreign bom. Five of the six developments (Seattle is the excep- 
tion) are located in tracts in which 37 percent to 62 percent of families were living in poverty. 
These rates are generally well above the 30 percent or 40 percent threshold commonly used to 
designate “high-poverty” areas. 
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Comparing the poverty rates of the census tracts containing the developments operat- 
ing the Jobs-Plus program and the poverty rates of the surrounding counties reveals that most 
of the Jobs-Plus tracts were essentially islands of high poverty in their respective regions, 
which were not particularly poor. This is starkly evident in Panel A of Figure 2. 1 . For exam- 
ple, in Baltimore, whereas 43 percent of families in the Jobs-Plus census track were living in 
poverty, only 14 percent of families in the surrounding county were poor. 1 A similar if less 
extreme pattern holds with unemployment rates in four of the six sites (Figure 2.1, Panel B). 
Again using Baltimore as an example, the 21 percent rate of unemployment of the Jobs-Plus 
census tract is three times higher than the surrounding county’s unemployment rate (7 per- 
cent). However, the contrast between local and countywide unemployment is much lower in 
St. Paul and only marginal in Seattle. 2 

The 2000 Census also reveals that rental vacancy rates varied across locations in the 
demonstration. For example, vacancy rates were highest in the census tracts containing the 
Jobs-Plus housing developments in Baltimore and Dayton (16 percent and 14 percent, respec- 
tively) and much lower elsewhere (see Appendix Table A.2). As Chapter 3 discusses, the Jobs- 
Plus developments in Baltimore and Dayton had the highest move-out rates among residents 
during the demonstration. 

Longer-term data on unemployment rates in the counties surrounding the Jobs-Plus 
sites offer another perspective on the context in which Jobs-Plus was implemented. Figure 2.2 
presents the long-term trends in unemployment rates, covering the six years before Jobs-Plus 
began operating in 1998 (the “baseline period” used in the impact analysis in Chapter 4) and six 
years afterward. As the graphs for each site indicate, unemployment rates varied markedly 
across locations at the start of this period, ranging from about 5 percent in St. Paul to about 10 
percent in Los Angeles. Reflecting the national economic boom over the course of the 1990s 
(see Chapter 1), the rates dropped substantially in all sites. By 1998, they ranged from around 3 
percent in St. Paul to about 7 percent in Los Angeles. By 2001, when Jobs-Plus was well under 
way, all sites had unemployment rates of around 5 percent or lower. From there on, however, 
they began to climb with the onset of the national recession and rose to about 7 percent in Day- 
ton, Los Angeles, and Seattle, where they stayed until the end of 2003, the last year of the dem- 
onstration’s operational phase. Unemployment rates also rose in Baltimore, Chattanooga, and 
St. Paul, but to no higher than about 5 percent. As Chapter 4 shows, the labor market experi- 
ences of residents in the Jobs-Plus and comparison developments track these broader trends to 
an important degree. 



'in Baltimore, the estimates for the surrounding comity include both Baltimore Comity and Baltimore City. 
“A more detailed analysis in Seattle that looks at blocks within census tracts suggests that poverty rates 
were about 45 percent in the blocks encompassing the Jobs-Plus housing development. 
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The Jobs-Plus Demonstration 
Figure 2.1 



Rates of Poverty and Unemployment in 2000 in the Census Tracts 
of the Jobs-Plus Development and Surrounding City and County, 

by Site 





Baltimore Chattanooga Dayton Los Angeles St. Paul Seattle 



SOURCE: MDRC calculations using data from the U.S. Bureau of the Census, Census 2000 Summary File 3. 

NOTE: “With the exception of Baltimore, each county includes its central city. Baltimore County, however, is 
not inclusive of Baltimore City. To make Baltimore County data comparable to county data for the other sites, 
poverty and unemployment rates for Baltimore City and Baltimore County were combined. 
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Unemployment Rate (%) 



The Jobs-Plus Demonstration 
Figure 2.2 

Local Unemployment Rates for the Jobs-Plus Sites from 1992 to 2003 
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SOURCE: U.S. Department of Labor, Bureau of Labor Statistics, Local Area Unemployment Statistics, 
http://data.bls.gov; data extracted on Lebruary 23, 2005. 
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Characteristics of the Jobs-Plus Developments and Residents 

All the housing developments where Jobs-Plus was implemented comprise mainly low- 
rise units (in contrast to the popular image of public housing as agglomerations of high-rise 
towers). All but one are relatively large, however — each with more than 400 households in 
residence. Several have a particularly good appearance, while others convey greater age and 
disrepair. And while some are close to commercial districts via public transportation, others are 
more isolated. 

Table 2.1 shows the characteristics of residents of the six housing developments that 
operated Jobs-Plus, who are part of the main research sample for evaluating the program’s im- 
pacts. This sample includes 2,259 individuals who were living in the Jobs-Plus developments in 
October 1998, the year that Jobs-Plus operations began; hence, they are referred to as members 
of the “1998 cohort” of residents of the Jobs-Plus developments. At that time, all these sample 
members were between ages 21 and 61 and were not classified as disabled by the housing au- 
thority. Although the full “target group” for the Jobs-Plus program extended to residents as 
young as age 18, the impact analysis (for methodological reasons explained in Chapter 4 and 
Appendix D) focuses only on those between ages 21 and 61, as does this entire report. (Another 
2,807 individuals with similar characteristics lived in the comparison developments, for a total 
1998 cohort sample of 5,066. For their characteristics, see Appendix Table A.3.) 

Three developments operating the Jobs-Plus program (in Baltimore, Chattanooga, and 
Dayton) were populated almost exclusively by African-Americans, who made up 94 percent or 
more of the Jobs-Plus sample for those sites. Residents of the other Jobs-Plus developments 
were a more varied ethnic mix, including large numbers of Hispanic and Asian/Pacific Islander 
households and substantial numbers of immigrants. For example, one of these developments (in 
Los Angeles) has predominantly Hispanic residents (79 percent of the sample), primarily of 
Mexican or Central American descent, but it also houses a sizable minority of Southeast Asians 
(12 percent), particularly from Vietnam and Cambodia. In another development (in St. Paul), 
Hmong and other immigrant families from Southeast Asia predominate (56 percent). And a 
third development (in Seattle) houses a wide mix of residents from different East African and 
Southeast Asian countries, as well as native-born U.S. citizens, with residents speaking as many 
as 22 different languages. 

The Jobs-Plus developments also varied with respect to other demographic characteris- 
tics, such as the percentage of females (ranging from 65 percent to 91 percent of the sample) 
and households with two or more adults (ranging from 14 percent to 74 percent). The latter 
households were most highly represented in the sites with high proportions of immigrants (Los 
Angeles, St. Paul, and Seattle), which included many married couples. 



23 




The Jobs-Plus Demonstration 

Table 2.1 



Selected Characteristics of Targeted Residents Aged 21 to 61 
Living in the Jobs-Plus Developments in 1998 



Characteristic 


Baltimore 


Chattanooga 


Dayton 


Los Angeles 


St. Paul 


Seattle 


All 

Developments 

Combined 


Race/ethnicity a (%) 


White (non-Hispanic) 


0 


4 


1 


1 


3 


8 


3 


Black (non-Hispanic) 


99 


94 


97 


6 


21 


32 


58 


Hispanic 


1 


2 


0 


79 


4 


0 


14 


Asian/Pacific Islander 


0 


0 


0 


12 


56 


28 


16 


Other 


0 


0 


0 


0 


0 


14 


3 


Missing 


0 


0 


2 


1 


16 


19 


7 


Household head (%) 


91 


93 


93 


57 


66 


70 


78 


Female (%) 


81 


91 


86 


65 


66 


71 


77 


Age a (%) 


21-24 years 


13 


27 


26 


12 


13 


12 


17 


25-34 years 


33 


38 


42 


28 


40 


33 


36 


35-61 years 


55 


35 


32 


59 


47 


55 


47 


Average age (years) 


37 


32 


32 


39 


36 


37 


35 


Lived in a household with (%): 


Two or more adults 


16 


15 


14 


74 


67 


54 


40 


No children 


45 


17 


27 


26 


5 


23 


24 


Children ages 


0-5 years 


25 


44 


47 


29 


56 


42 


40 


6-17 years 


44 


64 


48 


63 


88 


62 


61 


Sample size 


367 


282 


351 


513 


312 


434 


2,259 



(continued) 




Table 2.1 (continued) 



SOURCE: MDRC calculations using data from housing authority tenant (50058) records. 

NOTES: This sample (referred to as the "1998 cohort") includes all residents of each development in October 1998 who were between 21 and 61 
years old and not listed as disabled on public housing authority records. Because of missing data, the sample size for each characteristic may vary. 
In the average for all developments combined, the results for each housing development are weighted equally. 
a Due to rounding, distributions may not sum exactly to 100 percent. 




Turnover in the tenant populations was considerable over the course of the demonstra- 
tion. For example, for the six developments combined, 42 percent of the 1998 cohort had 
moved out of the Jobs-Plus developments over the subsequent three years. 3 

Work, Welfare, and Community Life When Jobs-Plus Was 
Launched 

The results of the baseline survey of residents — administered to household heads near 
the time that Jobs-Plus began 4 — offer additional insights into the types of people and places 
that Jobs-Plus was working to change. They show that the Jobs-Plus developments were places 
where residents had a stronger connection to the labor market than had been anticipated, the 
work was low-paying, reliance on welfare was high, poverty was widespread, and living condi- 
tions were difficult (see Table 2.2). 

Across the sites, 69 percent of the heads of household reported having had some em- 
ployment experience within the 12 months prior to the survey. Among all respondents, 51 per- 
cent said that they had worked in a full-time job during that period; 18 percent said that they had 
worked only in a part-time job. The work was not always steady, however. A somewhat lower 
proportion (57 percent) were working at the time they were interviewed. Most of the respon- 
dents who had worked in the recent past had done so in jobs that paid very low wages — $6.7 1 
per hour, on average — and only slightly more than half of the jobs (53 percent) provided im- 
portant fringe benefits. 5 

Longer-term trend data from Unemployment Insurance (UI) wage records presented in 
Chapter 5 indicate that the rates of employment in these developments had been climbing stead- 
ily (and unexpectedly) over the course of the 1990s, most likely in response to expanding job 
opportunities fueled by a growing economy. Thus, by the time Jobs-Plus operations began, 
these rates had already exceeded the 30 percent cap on the proportion of families employed that 
had been set as a condition for site selection. 

Table 2.2 also shows that residents faced an array of personal and situational problems 
that could make steady work and job advancement difficult. For example: 

• Limited education. Over half (56 percent) of survey respondents had no 
high school diploma or General Educational Development (GED) certificate. 



3 Chapter 3 presents a more detailed discussion of residents’ mobility patterns, and Chapters 4 and 5 con- 
sider the implications of resident mobility for Jobs-Plus’ s effects on work outcomes. 

4 The baseline survey was administered in 1998 in all sites except St. Paul, where it was conducted in 1999. 
For further information on the baseline and follow-up surveys of residents, see Appendix C. 

5 For further detail on residents’ prior employment experiences, see Martinez (2002). 
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The Jobs-Plus Demonstration 
Table 2.2 

1998 Baseline Survey (Household Heads) 



Selected Characteristics of Residents of the Jobs-Plus Developments 



Characteristic 


Baltimore 


Chattanooga 


Dayton 


Los Angeles 


St. Paul 


Seattle 


All 

Developments 

Combined 


Female (%) 


91 


97 


92 


95 


70 


83 


88 


Married and living together (%) 


6 


6 


5 


42 


44 


25 


21 


No high school diploma or GED (%) 


43 


46 


33 


68 


94 


53 


56 


Prior and current employment 


Ever employed (%) 


97 


94 


96 


88 


77 


77 


88 


Employed at time of interview (%) 


53 


60 


61 


60 


53 


53 


57 


Employed within the past 12 months (%) 


68 


78 


77 


66 


67 


59 


69 


Full time (%) 


43 


62 


60 


43 


56 


43 


51 


Part time (%) 


25 


16 


17 


23 


10 


16 


18 


If employed: 

Average hourly wage ($) 


6.09 


5.34 


6.19 


6.51 


8.72 


7.42 


6.71 


Received any fringe benefits (%) 


42 


55 


50 


44 


63 


63 


53 


Concerns associated with working full time (%) 


Making sure children are okay while at work 3 


19 


17 


27 


42 


NA 


33 


27 


Rent would be raised because making too 


much money 


53 


57 


57 


37 


46 


41 


48 


Losing benefits because making too much 


money 


32 


32 


35 


22 


23 


26 


28 


Receipt of public benefits in the past 12 months 3 (%) 


Anyone in household received welfare 13 


48 


54 


59 


50 


NA 


44 


51 


Anyone in household received food stamps 


69 


85 


72 


56 


NA 


56 


68 



(continued) 




Table 2.2 (continued) 



Characteristic 


Baltimore 


Chattanooga 


Dayton 


Los Angeles 


St. Paul 


Seattle 


All 

Developments 

Combined 


Estimated yearly household income (%) 


$5,000 or less 


57 


68 


48 


33 


8 


28 


40 


$5,001 -$10,000 


21 


19 


26 


32 


35 


41 


29 


$10,001-$15,000 


14 


6 


16 


20 


24 


21 


17 


More than $15,000 


8 


6 


9 


15 


33 


10 


14 


Health condition limits work or type of work (%) 


29 


29 


25 


28 


35 


35 


30 


Victimization (%) 

Victim of violence, robbery, or burglary 
or threatened with a weapon at the development 


within the past 12 months 


29 


42 


37 


17 


24 


23 


29 


Reported that the following were "pretty big" 
or "very big" problems at the development (%) 


People selling or using drugs 


81 


58 


74 


49 


15 


34 


52 


Guns and gunfire 


67 


39 


68 


49 


18 


27 


45 


Gangs causing trouble 


32 


24 


24 


56 


33 


27 


33 


Rated their housing development as a “good,” 


“very good.” or “excellent” place to live (%) 


59 


73 


60 


83 


89 


91 


76 


Reported they traveled outside the neighborhood 


at least once a week (%) 


84 


90 


91 


90 


85 


90 


88 


Sample size 


219 


220 


241 


237 


151 


160 


1.228 



SOURCE: MDRC calculations using data from the 1998 baseline survey. 



NOTES: In the average for all developments combined, the results for each housing development are weighted equally. 
a Respondents in St. Paul were not asked these survey questions. 
b Includes cash aid through AFDC/TANF and General Assistance. 




• Lack of adequate child care or child supervision while at work. Twenty- 
seven percent of respondents with children under age 18 indicated that full- 
time work would pose a significant problem for them because they would be 
concerned about whether their children would be okay. 

• Health or medical problems. Thirty percent said they had health conditions 
that would make it hard for them to work or would limit the amount or kind 
of work they could do. 

• Expectation of little or no economic improvement from employment. 

Forty-eight percent of respondents expressed concern about having their rent 
raised if they earned too much, while 28 percent believed that earning too 
much would cause them to forfeit benefits they had been receiving. 

Given the prevalence of unsteady and low-paying jobs, it is unsurprising that a large 
proportion of households had to rely on public assistance to help them get by. Indeed, 5 1 per- 
cent of all heads of household reported using Aid to Families with Dependent Chil- 
dren/Temporary Assistance for Needy Families (AFDC/TANF) or General Assistance during 
the prior year, and 68 percent reported using food stamps. 

Residents saw many problems — but also strengths — in their public housing devel- 
opments. For example, they voiced a variety of concerns about their personal safety and the 
public housing environment in general. Many experienced or feared apartment break-ins or 
thefts of personal property and identified the presence of guns and drugs as significant prob- 
lems. Across the sites, 29 percent had been victimized by an actual or attempted robbery, bur- 
glary, or violent attack or threat within the year prior to the survey. Moreover, 52 percent said 
that people selling or using drugs was a “pretty big” or “very big” problem in their develop- 
ment, and 45 percent said the same about guns and gunfire. In addition to diminishing the over- 
all quality of life in the developments, these safety concerns may have caused residents to fear 
traveling to work, especially if it required commuting by public transportation, and late at night. 

On these measures, some differences among the sites are striking. For example, as Ta- 
ble 2.2 shows, crime victimization rates ranged from 17 percent in Los Angeles to 42 percent in 
Chattanooga. Moreover, while 15 percent of respondents in St. Paul said that “people selling or 
using drugs” in their developments was a “pretty big” or “very big” problem, 81 percent gave 
that response in Baltimore. According to the evaluation’s field research, much of the problem in 
Baltimore centered on use and sale of heroin, while Dayton had a greater problem with crack 
cocaine. And while 18 percent of St. Paul respondents said that “guns and gunfire” were a 
“pretty big” or “very big” problem in their developments, over two-thirds gave this response in 
Baltimore and Dayton. Concerns with “gangs causing trouble” stood out in the Los Angeles 
site, where 56 percent of respondents identified this as a “pretty big” or “very big” problem. 
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Overall, crime and safety concerns appeared lowest in St. Paul and Seattle — a pattern consis- 
tent with observations made by the field researchers. 

Yet, across the sites, these and other problems did not lead residents to adopt a com- 
pletely negative attitude toward their developments. Indeed, 76 percent said that their develop- 
ment was a “good,” “very good,” or “excellent” place to live (with “good” being the most 
common rating), although this positive assessment was lowest (about 60 percent) in Baltimore 
and Dayton. And few residents reported feeling isolated — in fact, 88 percent said that they 
traveled at least once a week outside the neighborhood where their development is located. 
These more positive features of the developments were all foundations on which Jobs-Plus 
could build to help public housing communities better encourage and support employment. 



Comparability of the Program and Comparison Developments 

As Chapter 1 explains, the evaluation’s research design involved identifying in each city 
two or more housing developments that had roughly similar types of people. One of these devel- 
opments was then randomly selected to operate Jobs-Plus, and one or two were allocated to a 
comparison group. Because random assignment was conducted at the development level — and 
for only a few developments within each city — the samples of individuals in the program and 
comparison groups were not expected to have perfectly matching demographic profiles. As it 
turns out, however, the quality of the match is quite good overall, as shown in Appendix Table 
A.3. Furthermore, responses from the residents (household heads) who were interviewed as part 
of the baseline survey administered at the start of Jobs-Plus shows that, on a variety of dimensions 
(such as perceptions of problems in the development, concerns about crime and safety, assess- 
ments of the quality of life in the developments, and employment rates), the program and com- 
parison developments were remarkably similar. 6 And in sites where the differences appear some- 
what larger, there is little indication that conditions were consistently more favorable in the Jobs- 
Plus than the comparison developments; if anything, they were somewhat worse. 7 Furthermore, as 
Chapter 4 illustrates, prior to the start of Jobs-Plus, the employment and earnings trends of resi- 
dents in the 1998 cohort who were living in the Jobs-Plus development were extraordinarily simi- 
lar to the trends of those living in the comparison developments. Together, these findings provide 
strong assurance that the impact analysis, when comparing subsequent outcomes for those two 



6 Table 6.5 in Chapter 6 helps supports this point in three sites (Baltimore, Chattanooga, and Dayton). Sup- 
porting evidence from other sites and across other measures is available in tabulations presented in unpublished 
baseline survey “data resources books” compiled by MDRC. 

7 For example, the survey data and field research interviews in Baltimore suggest that crime was a more 
pressing problem at the Jobs-Plus development than at the comparison developments. At the same time, it is 
important to note that the pre-Jobs-Plus employment and earnings trends are remarkably similar, suggesting 
that the crime and safety problems did not lead to differences in labor market participation. 
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groups of residents, does so for essentially the same types of people — a necessary condition for 
safely attributing any difference in those outcomes to Jobs-Plus. 8 



Summary and Conclusions 

This chapter shows that the developments selected to operate Jobs-Plus all housed 
very poor tenants with significant impediments to employment. Although their residents had 
more work experience than had been expected, large numbers were unemployed or underem- 
ployed, and many who were working had jobs that would not lift them out of poverty. Thus, 
there remained considerable room for Jobs-Plus to improve their employment rates and (espe- 
cially) their average earnings. Still — and to their credit — residents were already responding 
to the expanding job opportunities spawned by an improving economy across all the sites be- 
fore Jobs-Plus began. 

Despite their many similarities, the Jobs-Plus developments were distinguished by a va- 
riety of local conditions and population characteristics that would present the sites with different 
kinds of challenges in implementing the program. Some, for example, would have to adapt their 
operating strategies to suit the different cultural orientations of diverse immigrant groups and 
native-born residents; some would have to contend with social environments in which crime 
and safety issues and extensive drug dealing (often emanating from outside the development) 
were a daily reality; and, with differences in local rental vacancy rates in the broader commu- 
nity, some would have to adapt to a frequently changing tenant population, while others worked 
with a more stable group. 

Chapter 3 examines how the sites implemented their Jobs-Plus programs across these varied 
settings. It thus considers whether the model is a feasible one to operate under a range of circum- 
stances that can be found among many other public housing developments across the nation. 



Data from the 2000 Census (see Appendix Tables A.l and A.2) show that, in each city, the match be- 
tween the characteristics of the broader population of census tracts encompassing the Jobs-Plus and compari- 
son developments is not nearly as close as the match between the residents of the housing developments them- 
selves. However, the comparison group tracts, like the Jobs-Plus tracts, were generally high-poverty areas. 
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Chapter 3 

The Implementation Experience 



Jobs-Plus was an ambitious program to implement. Planning and delivering the interven- 
tion through new interagency and resident partnerships, bringing employment assistance directly 
into public housing developments, crafting new rent mles to make work pay, fostering neighbor- 
to-neighbor support for work, and offering assistance on a “saturation” basis — that is, to all 
working-age, nondisabled residents of the developments — pushed well beyond the scope of typi- 
cal self-sufficiency initiatives in public housing. Thus, it was not at all clear at the outset of the 
demonstration that it would be feasible to operate the Jobs-Plus model in the real world. 

This chapter shows that, in the end, most of the sites did a good job of implementing the 
program, although not without difficulty. The challenges varied across the sites, but, in general, 
the sites had to work hard to create a new culture of collaboration among housing authorities, 
social service agencies, and residents; sustain the active support of senior housing officials; 
overcome cumbersome personnel and procurement policies of the housing agencies; appeal to 
residents of different racial and ethnic groups; deal with enduring issues of safety, substance 
abuse, and other family crises; and contend with high resident turnover. In addition, Jobs-Plus at 
first encountered widespread skepticism from residents, who had often seen programs of vari- 
ous kinds come and go. This was not helped when, due to federal funding problems, a central 
component of the program — the rent-based financial work incentives — was not put into place 
in most sites for well over two years after site selection. 

In time, four of the six sites were able to build coherent programs of reasonable quality 
(even if not all the components were well integrated), making the demonstration a “fair test” of 
the Jobs-Plus model (see Table 1.1 in Chapter 1). Although the program was voluntary, large 
numbers of residents decided to participate actively, and, by a variety of measures, Jobs-Plus 
“touched” many other residents as well. 

This chapter recounts the sites’ major strategies, difficulties, and accomplishments in 
implementing Jobs-Plus, drawing heavily on earlier MDRC reports that have examined the pro- 
gram’ s operations in much greater depth. 1 By showing how Jobs-Plus operated in very different 
local settings and how residents responded to it, the chapter sets the stage for understanding the 
program’s impacts on residents’ employment, earnings, and welfare receipt. 



See, for example. Bloom, 2000; Gardenhire-Crooks, 2004; Kato, 2003a, 2003b, 2004; Kato and Riccio, 
2001; Liebow, 2004; Miller and Riccio, 2002; and Riccio, 1999. 
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Launching Jobs-Plus 

The participating housing authorities and their partners began operating some elements 
of Jobs-Plus in 1998. The program offices were located in converted housing units, community 
centers, or other facilities that provided a convenient place for program staff and residents to 
meet. It was universally accepted by the sites that, as a place-based initiative, Jobs-Plus must 
have a strong physical presence, which was essential in helping staff to feel like and be seen as a 
vital part of the community they were serving. 

The program offices — which functioned in many respects like on-site “job centers” — 
opened their doors about a year after being selected for the demonstration. However, it took an- 
other two years or so (until late 2000, which was much longer than had been hoped) before 
Jobs-Plus could evolve into a mature intervention that reflected much of the designers’ original 
vision, and getting there was a complicated journey. 

One problem contributing to the long and difficult gestation period was the inevitable 
struggle involved in establishing new collaborative forms of decision-making. As explained in 
Chapter 1, responsibility for the detailed design of and oversight over the program in each site 
rested not with the local housing authority alone but with an interagency and resident partner- 
ship, or “collaborative.” Each collaborative included representatives of the public housing au- 
thority, the welfare department, the workforce development agency, public housing residents, 
and other local organizations — particularly nonprofit social service and employment and train- 
ing providers. In each collaborative, the local housing authority functioned as the “lead agency.” 

The sites struggled to build efficient and effective collaboratives. Although most began 
with aspirations of sharing decision-making responsibilities and building a sense of collective 
“ownership” over Jobs-Plus — goals embraced also by the demonstration’s designers — these 
goals proved very difficult to achieve in practice. In fact, the collaboratives got bogged down 
for long periods of time as they sorted out how to make decisions and how the lines of authority 
and accountability should flow. Moreover, in some sites, tensions often flared between the resi- 
dents and the representatives of the housing authority — fueled partly by preexisting adversarial 
relationships — as they struggled to define the boundaries between resident empowerment and 
housing authority control. In addition, managing the collaboratives typically fell to the Jobs-Plus 
project directors, and this competed with the time that they needed to spend on program activi- 
ties. Although it is difficult to imagine that Jobs-Plus could have operated well without some 
form of collaboration among the housing authority, other local agencies, and residents, it is also 
fair to say that probably none of the partners would go about the task of collaboration in exactly 
the same way again, for they themselves were frustrated by the pace of progress in getting a 
fully functioning Jobs-Plus program designed and implemented. Yet, despite these problems, 
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the collaboratives in most sites continued to function — even if in modified form — as active, 
cooperating entities through to the end of the demonstration. 2 

Another source of complication and delay in launching Jobs-Plus was the fact that the 
housing authorities and their partners had to build a new program from “scratch” at each partici- 
pating development. This meant solving fundamental problems of infrastructure facing any new 
program, such as designing and filling new staff positions and arranging for space in which to op- 
erate the program. In addition, each collaborative had to design and implement its own package of 
job search, education, training, and support services, along with the program’s other two compo- 
nents, financial work incentives and community supports for work. (The choices they made are 
described in a later section.) Although MDRC provided technical assistance to help the sites make 
their design decisions, the expectation was that specific programmatic strategies would be de- 
signed locally rather than imposed centrally by MDRC or the demonstration’s sponsors. Finding 
the right balance between a “bottom-up” approach (led by the sites) and “top-down” approach (led 
by MDRC and the demonstration’s sponsors) proved difficult, however. In some sites, progress 
was impeded by staff members' limited time and capacity for such design work and by the slow 
process of making program design choices through the collaborative. At the same time, some lo- 
cal staff complained that they were not being given enough concrete design direction by MDRC 
soon enough, particularly in developing strategies for work-focused case management and for 
implementing the community support for work component. The latter component presented a 
special challenge to implement because it was the least well formulated, least precise element of 
the Jobs-Plus model at the start of the demonstration. 

The problems in launching Jobs-Plus were exacerbated by unanticipated delays in se- 
curing federal funding for the rent-based work incentives, which was beyond the control of the 
sites. Although the rent incentives had been widely expected to galvanize residents’ support for 
and involvement in Jobs-Plus, the incentives had to be approved by the U.S. Department of 
Housing and Urban Development (HUD), which would provide the funding needed to cover 
potential losses in rent revenues that the local housing authority might incur by permitting em- 
ployed households that participated in Jobs-Plus’ s rent incentives program to keep more of their 
earnings. However, issues arose between HUD and the Congressional committee that oversees 
HUD’s total departmental budget over how to cover the potential losses in rent revenues to local 
housing authorities. After many months of negotiations, the funding problem was solved in the 
spring of 2000. But, in the meantime, the long delay in implementing the rent incentives fueled 
widespread disappointment and cynicism among residents — and staff — and threatened to 



2 For a detailed study of the experiences of the Jobs-Plus collaboratives, see Kato and Riccio (2001). 
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undermine Jobs-Plus’ s credibility among the residents who had been eagerly anticipating rent 
incentives as a tangible benefit of participating in the program. 3 

In St. Paul, Jobs-Plus had begun to offer the rent incentives at the end of 1998, after re- 
ceiving HUD’s initial approval for its plan. Impressively, in an effort to salvage the program’s 
fledgling reputation, the housing authority made good on its rent incentives promise to residents 
even after HUD temporarily suspended approval, and it absorbed the costs of the rent reductions 
directly for the first year. Across all the sites, the rent incentives did not become available with 
full federal funding until mid to late 2000. By then, however, with guidance from MDRC and 
HUD, the sites had crafted a diverse and innovative set of policies for countering what were 
widely believed to be work-depressing effects of traditional public housing rent rules. 

Jobs-Plus thus did not begin at any of the demonstration sites as a fully formed program 
with the capacity to provide all three of the model’ s components to the residents. As illustrated by 
Figure 3.1, when Jobs-Plus opened its doors in 1998, the programs began enrolling residents and 
offering them some employment-related services, such as job search assistance and help in enter- 
ing education and training classes. The rent-based financial work incentives came on-line next. 
The community support for work component was generally the last piece of the Jobs-Plus model 
to be launched in a formal, structured way. (Although some sites, such as Dayton, began this 
component relatively early, it did not fully take shape anywhere until later in the demonstration.) 

In retrospect, this “staged” implementation, although not planned, may have had a posi- 
tive side to it. Because Jobs-Plus is a complex model for which there was no precedent to guide 
the sites in adapting it to their local circumstances, trying to operationalize all components si- 
multaneously may have added a significant extra burden to an already-difficult startup process. 

Because of the slow implementation process and many startup difficulties, the years 1998 
and 1999 have been defined for the purposes of the evaluation as a transitional, or “rollout,” pe- 
riod for Jobs-Plus. Consequently, as explained in Chapter 4, the program's impacts on employ- 
ment and earnings after that transitional period (that is, during the years 2000 to 2003) are consid- 
ered a better estimate of the potential of Jobs-Plus than its effects during the earlier years. 



Variation in the Sites’ Efforts to Operate a Quality Program 

Building and sustaining a viable Jobs-Plus program was a challenge for all the sites. 
However, the particular difficulties that they encountered were not the same everywhere, and 



3 Kato and Riccio, 2001. 



36 




The Jobs-Plus Demonstration 
Figure 3.1 

The Jobs-Plus Implementation Time Line 




Chattanooga 




Dayton 




Los Angeles 




St. Paul 




Refers to the availability of the employment-related services component at each site, beginning with the year 
when the local Jobs-Plus program opened an office and began assisting residents. Chattanooga scaled back 
these services in 2002. 



|Refers to the availability of the financial (rent) incentives component at each site, beginning with the year when 
Jobs-Plus could begin enrolling households into the incentives program. 



[ [ Refers to the availability of the community support for work component at each site, defined in this figure as 
the establishment of a formal cadre of volunteer resident outreach workers. Chattanooga had not fully 
implemented this component. Seattle included a range of other community-building activities under this component. 



NOTES: “Program Rollout Period” refers to the demonstration time period during which the sites had not 
implemented all of the Jobs-Plus components and were still developing the program flow and building the program 
staff. 

“Ongoing Activities” refers to the demonstration time period during which the full complement of Jobs-Plus 
components was generally in place across the sites (with the exception of Chattanooga). However, activities began to 
wind down at several sites around mid-2003. 
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they did not all operate programs of similar quality. Table 3.1 illustrates this variation by high- 
lighting the key efforts, setbacks, and achievements at each site as it struggled to make the Jobs- 
Plus vision a reality on the ground. 

Personnel and Procurement Issues 

The Jobs-Plus programs typically included the following staff positions. A project direc- 
tor managed the program’s daily operations. Case managers or job counselors guided and moni- 
tored residents’ efforts to prepare for, seek, and retain jobs and also connected residents to relevant 
on-site or off-site services. Job developers built program links with employers and identified job 
openings and arranged interviews for residents. Resident liaisons led efforts to tell residents about 
Jobs-Plus and get them involved in the program. These staff positions were filled by employees of 
the housing authority and employees from collaborating agencies (for example, the welfare office 
or other education, training, or social service providers) who were colocated at the Jobs-Plus of- 
fice. In some sites, residents were also hired for regular staff positions. 

In some cases, filling these positions with appropriate staff and in a timely way became 
problematic, in part because of the slow and cumbersome personnel and procurement regula- 
tions of the local housing authorities. For example, in Los Angeles during the first few years of 
the demonstration, chronic turnover of the Jobs-Plus project director and staff vacancies ham- 
pered the ability of the site to develop a smoothly functioning program, raising serious questions 
at the time about its continued viability. However, steps were taken to address these problems, 
and — with the sustained support of senior housing authority officials and development-level 
housing management staff — these efforts led to an impressive revival of the program, setting it 
on a sound operational footing by 2000. In a number of sites, program operations were hindered 
early on by other housing authority regulations. In particular, procurement rules significantly 
delayed purchasing and installing essential equipment, such as an automated management in- 
formation system (MIS) and the computers needed to enroll, assess, and track clients. 

In most sites, key program positions were eventually filled with competent staff, many 
of whom stayed with the program until the end of the demonstration. Personnel problems were 
most enduring in Chattanooga. Some of the difficulties encountered there arose from the col- 
laborative’s unusually strong commitment to the concept of a “resident-driven” Jobs-Plus pro- 
gram. While admirable, this vision led the collaborative to assign to key staff positions residents 
who were not fully prepared or trained for the roles they had to play in a complicated program- 
matic intervention. This contributed to the relatively weak employment intervention that 
emerged during the program’ s initial period. 
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The Jobs-Plus Demonstration 



Table 3.1 



Highlights of the Sites’ Experiences in Operating Jobs-Plus 



Site 


Operational Highlights 


Baltimore 

Gilmor 

Homes 


• Made a promising start with a full complement of staff and good relationships with an 
extensive network of local service agencies. Distinctive in its close coordination with a 
special on-site health office early on. 

• Jobs-Plus had peaked by the end of 2000. Reductions in funding following expiration of key 
grants starting in 2001 resulted in steady loss of staff, including case managers, colocated 
welfare caseworker, and job developer. Retrenchment of workforce services in local 
Sandtown-Winchester neighborhood eroded capacity of referral network to offer 
employment services to Jobs-Plus participants. On-site health office closed in 2002. 

• Rent incentives available in November 2000 but poorly administered by housing authority. 


Chattanooga 

Harriet 

Tubman 

Homes 


• Until 2000, Jobs-Plus foundered. Program was partially staffed by residents unprepared for 
their roles and given inadequate oversight by senior housing officials. Rent incentives were 
implemented in November 2000. 

• Program was reconstituted between June 2000 and June 2002 with improved staffing, 
employment counseling, service referrals, and management. Overall, however, progress in 
strengthening program quality remained limited. Community support for work component 
was never fully implemented. 

• Housing authority remained enthusiastic and cooperative in implementing the rent 
incentives. 

• Overall, housing authority gave low priority to Jobs-Plus after a change in housing authority 
leadership. Its focus on privatizing its property management and resident services operations 
made it unlikely that the agency could oversee Jobs-Plus adequately, hi the summer of 
2002, the housing authority and national demonstration partners agreed to continue a scaled- 
down, financial-incentives-only version of the program (although provision of some on-site 
services continued informally). 


Dayton 

DeSoto Bass 
Courts 


• Sankofa (a nonprofit agency that was formerly the housing authority’s Resident Services 
division) administered Jobs-Plus and provided stable and capable leadership and staff. 

• Enduring high-level support in securing funding and program services of the housing 
authority and other collaborative partners, including Montgomery County’s multiservice 
“one-stop” job center. 

• Program offered extensive outreach, short-term training, job readiness and search, intensive 
case management, colocated welfare caseworker, and on-line access to local one-stop’s 
employment database. First site to recruit residents for community support for work 
component ( building captains). 

• Rent incentives were implemented in May 2000 and actively marketed by the housing 
management office as well as Jobs-Plus. 



(continued) 



39 





Table 3.1 (continued) 



Site 


Operational Highlights 


Los Angeles 

William 
Mead Homes 


• Jobs-Plus was reconstituted in 2001 after a slow buildup due to housing authority-related 
staffing gaps, leadership turnover, and equipment needs. Became strong program that 
provided on-site job search, GED classes for Spanish speakers, and training, with intensive 
outreach and case management as well as on-site welfare-to-work caseworker and job 
developer from other public agencies. 

• Rent incentives were implemented in June 2000 with strong support from and active 
promotion by housing management staff. 

• Program had strong community support for work component. Beginning in November 2000, 
residents were hired and tr ained as community coaches to help Jobs-Plus publicize activities 
and job opportunities and recruit participants; played leading role in bringing basic education 
classes on-site. 


St. Paul 

Mt. Airy 
Homes 


• Program had to address special language- and immigrant-related barriers to outreach, service 
delivery, and employment. 

• Offered on-site job counseling, job clubs, and case management, some customized short- 
term training classes, U.S. citizenship classes, and ESL and GED instruction. Hrnong 
Women's Support Group assisted with mental health and cultural issues. Head Start and 
after-school and summer programs were available for children and youth. Education and 
training were offered through referrals to local schools and agencies. Had consistent, 
professional staffing and colocation of staff from partner agencies. 

• First site to implement rent incentives, beginning in November 1998, using housing 
authority funds. Strong management office support in recruiting, orienting, and 
enrolling households for Jobs-Plus as well as administering rent incentives. 

• Residents volunteering as community outreach workers who spoke languages of ethnic 
groups at Mt. Airy provided important assistance in publicizing Jobs-Plus activities and job 
opportunities. 



(continued) 
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Table 3.1 (continued) 



Site 


Operational Highlights 


Seattle 

Rainier Vista 

Garden 

Community 


• Program had strong and stable staff; built solid relationships with the development's highly 
ethnically diverse population, who collectively spoke 22 languages. Provided on-site job 
counseling and case management and ESL and financial management classes, in 
partnership with Refugee Women's Alliance. Off-site referrals for education, training, and 
support services. 

• Housing authority received HOPE VI grant to tear down and rebuild the Jobs-Plus 
development, in stages. Unique context of HOPE VI led to site’s formal withdrawal from 
national demonstration at the end of 1999. However, Jobs-Plus continued to operate with a 
broader target group and expanded mission that included helping residents deal with issues 
related to relocation. 

• Resident relocation process was under way from 2000 to mid-2002 as part of the first stage 
of redevelopment. 

• Rent incentives were implemented in September 1999 with strong housing authority 
support. However, enrollment in this component closed in April 2001 with phasing in of 
HOPE VI. 

• With declining numbers of residents at Rainier Vista, intensity of services on-site declined 
as staff were assigned additional responsibilities to provide services to residents of other 
developments. Few services were provided to residents once they relocated out of public 
housing. 

• Residents who spoke languages of ethnic groups at Rainier Vista were hired and trained as 
resident outreach and orientation specialists to publicize Jobs-Plus services and job 
opportunities and to recruit participants. 



SOURCES: MDRC field research and program documents. 

NOTES: “GED” refers to General Educational Development; “ESL” refers to English as a Second 
Language. 
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Housing Authority Support 

In general, strong support for Jobs-Plus from senior housing authority officials and col- 
laborative partners was impressive, but it was not sustained everywhere over the course of the 
demonstration. In four sites — Dayton, Los Angeles, St. Paul, and Seattle — that support was 
strong and unwavering. In fact, because of that commitment, these same four sites continued 
operating a version of Jobs-Plus after the end of the demonstration (although without the special 
rent incentives). 4 This is a testament to their belief in the program’s potential to make an impor- 
tant difference in the lives of residents. 

The picture was much different in Chattanooga and Baltimore. In Chattanooga, the ex- 
ecutive director of the housing authority at the outset of the demonstration was an avid sup- 
porter of Jobs-Plus. However, those who succeeded him had limited interest in the demonstra- 
tion and were increasingly preoccupied by other housing authority initiatives — most impor- 
tantly, a plan to transfer housing management responsibilities to a private company. Conse- 
quently, mounting managerial and staffing problems at Jobs-Plus in Chattanooga were not ade- 
quately addressed, and residents’ confidence and participation in the program declined precipi- 
tously, bringing program activity in Chattanooga to a virtual standstill during the first half of 
2000. Over the course of the next year, with extensive technical assistance provided by MDRC, 
efforts were made to rebuild and strengthen the Chattanooga program. New staff were hired; 
roles were redefined; and a new management structure was devised. Steps were also taken to 
build up the community support for work component, which, up to that point, had never really 
gotten off the ground. Despite these efforts, further progress was limited, and a decision was 
finally reached between MDRC, the Jobs-Plus collaborative, and the demonstration’s main fun- 
ders to restructure the program in the spring of 2002 to focus primarily on financial incentives. 5 
Still, some employment-related assistance did continue, including referrals of residents to part- 
ner agencies for help with job placement, education, and training. 

In Baltimore, although Jobs-Plus enjoyed continued support by some housing authority 
administrators, turnover among the top officials, in conjunction with other pressing issues con- 
fronting the housing authority, led those officials to give much less attention to Jobs-Plus than it 
needed. 6 On matters where their help was particularly critical — such as the implementation of 



4 For example, the housing authority in Dayton received funding from HUD under the Resident Opportunities 
for Self-Sufficiency (ROSS) grant program to extend aspects of Jobs-Plus services to all developments. This fund- 
ing lasts through 2006. 

5 For a more detailed account of the evolution and transition of the Chattanooga program, see Bowie (2003). 

6 Some of the same problems, including an audit by HUD, that contributed to the housing authority’s delay 
in implementing the provisions of the 1998 federal housing legislation (QHWRA), which did not occur until 
2003, diverted attention from Jobs-Plus. 
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the rent-based financial work incentives (which depended on the cooperation of the housing 
management office) — efforts fell short, as explained later in this chapter. 

Crime and Safety Issues 

Serious problems with safety and crime undercut program operations at a few sites. In- 
deed, over the course of the demonstration, Los Angeles had to contend with a number of gang- 
related problems and two fire-bombings, which damaged a housing authority office and van; 
Los Angeles and Baltimore experienced a number of shootings at the Jobs-Plus developments 
(some fatal); and both developments, along with Dayton, continued to struggle with a very ac- 
tive drug trade (which was most pronounced in Baltimore). These problems were generally less 
severe in St. Paul and Seattle. 

Other Challenges 

The sites also had to contend with a variety of other types of challenges. For example, 
in the multiethnic developments (Los Angeles, St. Paul, and Seattle), the diversity of residents’ 
cultural backgrounds and languages required efforts to provide assistance appropriate to their 
distinctive needs and circumstances. 7 High resident turnover in Baltimore, Chattanooga, and 
Dayton (which is discussed below) required Jobs-Plus continuously to direct staff and resources 
toward outreach efforts to inform incoming residents about the program. And, like many other 
programs, Jobs-Plus faced the difficult challenge of having to address the complex needs of 
some residents who struggled with substance abuse and various family crises. 

The Baltimore Jobs-Plus program, in addition, confronted a severe and unanticipated 
retrenchment in funding and service access that ultimately compromised the quality of its pro- 
gram. Although it began the demonstration with substantial funding, with a full and dedicated 
staff, and with strong ties to the extensive network of service providers in the Sandtown- 
Winchester neighborhood where the Jobs-Plus housing development (Gilmor Homes) was lo- 
cated, many of these advantages began to dissipate just as the program was poised to enter a 
steady-state period of operations. Starting around 2000, funding losses, staff attrition, and reduc- 
tions in services offered by partner agencies in Sandtown-Winchester led to a steady decline in 
the operational capacity and quality of Jobs-Plus. The tragic death of the Jobs-Plus office man- 
ager at the end of 2001 — which had a devastating effect on staff morale — compounded these 
problems. Indeed, just as Los Angeles and other sites were ramping up their Jobs-Plus pro- 
grams, the Baltimore program, which had made a promising start, had already peaked. 8 



7 Kato, 2002. 

8 For example, in 2001, the Enterprise Foundation, a key local funder, began to phase out its funding of 
workforce initiatives in Sandtown-Winchester. Moreover, the impressive network of about 20 agencies provid- 

(continued) 
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HOPE VI Relocation and Redevelopment in Seattle 

The Jobs-Plus program in Seattle operated in very special circumstances. In addition to 
having the most ethnically diverse tenant population, which collectively spoke 22 different lan- 
guages, it had to adapt to the advent of a HOPE VI redevelopment plan. As explained in Chap- 
ter 1, this plan introduced a confounding contextual factor that was unique among the Jobs-Plus 
sites. The plan called for the housing development in which Jobs-Plus was located (Rainier 
Vista) to be demolished and replaced by a larger complex that would house a population with a 
broader mix of incomes, including subsidized and unsubsidized tenants. Demolition would pro- 
ceed in stages, with one-half of the development being tom down and rebuilt before the focus 
shifted to the remainder of the property. The process of relocating residents to the other side of 
the development, other public housing developments, or private units (with Section 8 subsidies) 
began in 2000 and was completed by mid-2002, with the first phase of demolition beginning 
later that year. 9 

Because the HOPE VI plan called for completely rebuilding the physical space and social 
community in which Jobs-Plus was located — and in ways that were not comparable to any in the 
other Jobs-Plus sites — Seattle was withdrawn from the national demonstration, and MDRC’s 
technical assistance was terminated at the end of 1999. Nonetheless, the housing authority contin- 
ued to operate Jobs-Plus in modified form as the centerpiece of its efforts to provide residents with 
continuing self-sufficiency services throughout the redevelopment process. In doing so, it broad- 
ened the target group to include all residents of the development, not just the working-age, non- 
disabled residents originally targeted by Jobs-Plus. It also added relocation-related assistance to its 
menu of services. However, new enrollments in the rent incentives component were ended by 
March 2001, 10 and, contrary to what had been intended, little service was offered to residents who 
were relocated to off-site housing. In general, the scope and intensity of employment assistance 
offered at Rainier Vista for the remaining residents lessened after the first stage of relocation was 



ing employment-related services in Sandtown- Winchester — a network on which Jobs-Plus depended for serv- 
ing its residents through referral — dwindled to about three providers by 2003. Visions for Health Consortium, 
a collaboration of health care agencies that also provided on-site medical screenings and referrals for residents 
of the Jobs-Plus development, was closed in 2002. Jobs-Plus itself saw its staff shrink after the expiration of 
two important HUD grants without new resources to replace them: an Economic Development and Supportive 
Services grant and a Drug Elimination grant. An added problem in Baltimore was that the resident council, 
which had a strong but elderly leader who passed away early in the demonstration, became largely nonfunc- 
tional as the demonstration progressed, leaving Jobs-Plus without the benefit of a stable formal resident leader- 
ship organization on which it could draw to enhance its outreach efforts. 

9 New construction of that part of the development was expected to continue into late 2005, at which point re- 
location, demolition, and reconstruction activities would begin on the other side of the development. 

10 Thus, members of the 1998 cohort who had not worked or taken advantage of the rent incentives by then 
could not obtain them after that date. Those who had taken up the rent incentives could continue to receive them 
through the end of 2003. 
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completed." MDRC continued to study the implementation and impacts of Jobs-Plus in Seattle, 
but (for funding reasons) the research agenda for this site was scaled back. 

Cross-Site Comparison 

In sum, three sites — Dayton, Los Angeles, and St. Paul — were able to contend best 
with the operational challenges they confronted, and Jobs-Plus owed much to the critical and 
sustained support it got from senior housing authority officials. Baltimore and Chattanooga had 
a more difficult time getting and keeping all the Jobs-Plus components in place. Finally, Seattle 
operated a strong program, especially during the first few years of the demonstration, but it had 
to contend with the special disruptions associated with the large-scale relocation of residents 
and reconstruction of the development. 



Scope of Services, Incentives, and Supports for Work 

Jobs-Plus’ s three components were intended to function as complementary ways of in- 
fusing public housing developments with a dramatic and unprecedented effort to promote 
work . 12 In time, the program became widely known in the participating developments as a place 
where residents could get help with their employment needs. The financial incentives were per- 
haps the most eagerly anticipated benefits of the new program. According to field research in- 
terviews, they were an important “hook” that grabbed residents’ attention and got many of them 
to come forward. However, many residents also looked to the program’s employment services 
component to help them find their way into work or into better jobs. 

A Broad Set of Employment-Related Services with an Emphasis on Help 
Finding Jobs 

A central thrust of Jobs-Plus at all sites was to help nonworking residents find work as 
quickly as possible — whether they were new to the labor force, detached from it for a long time, 
or only recently unemployed. The programs also aimed to help residents who were already work- 
ing but who hoped to find better jobs. Assistance with job search was thus a core program feature 
everywhere, and it was typically provided on an individual basis by employment counselors with 
the aid of the job developers, who helped to identify employers needing workers in positions for 
which Jobs-Plus participants would be suitable. In several sites, the job developers also helped 
prepare residents for job interviews, sometimes escorting them to their appointments. Some pro- 



1 'For a detailed account of the implementation of Jobs-Plus in the context of HOPE VI in Seattle, see Lie- 
bow et al. (2004). 

l2 See Appendix Tables B.l to B.6 for a site-by-site summary of the approaches to implementing the Jobs- 
Plus components. 
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grams supplemented these efforts by directly providing or referring residents to group-based job 
clubs, which taught job-hunting and interviewing techniques. Jobs-Plus also offered instruction 
(directly or through referral to other organizations) in “soft” employment skills, such as under- 
standing employers’ expectations and appropriate workplace behavior. 

For residents interested in education and training, Jobs-Plus offered help locating and en- 
rolling in suitable programs. In some cases, Jobs-Plus offered short-term basic education and cer- 
tain skills training classes (for example, certified nurse assistant training) on-site at the develop- 
ments. However, reflecting residents’ own interest in quick employment or new jobs, education 
and training activities were emphasized considerably less than direct job placement assistance. 

Jobs-Plus also offered a broad range of support services that could aid residents in find- 
ing and keeping jobs. These included assistance arranging and paying for child care and trans- 
portation and referrals to appropriate agencies for substance abuse treatment, counseling for 
family problems, and health services. In keeping with a broad vision of helping residents im- 
prove the quality of their lives, the programs also provided or facilitated access to a variety of 
life-skills classes, financial management workshops, driver’ s education, help with immigration- 
related problems in those developments with larger immigrant populations, help preparing tax 
returns and applications for the Earned Income Tax Credit (EITC), and “paperwork nights” to 
complete applications or documentation required by a variety of government programs. 

Flat Rents and Other Rent Reforms to Help Make Work Pay 

To encourage residents to take and keep the relatively low-paying jobs that they were 
likely to qualify for — as well as to encourage many of them to work more or to seek higher- 
paying jobs — Jobs-Plus included new rent policies that limited how much employed residents’ 
rent increased when their incomes grew. 13 Under traditional rent policies, residents paid 30 per- 
cent of their household’ s countable income in rent, up to a maximum amount tied to the cost of 
operating their public housing units. Under Jobs-Plus, residents paid less of their overall income 
in rent, and the savings could be substantial. 14 

Jobs-Plus rent incentives incorporated two main strategies, with different sites taking 
one or the other approach: (1) flat rents, which specify a fixed rental payment regardless of 

13 Miller and Riccio (2002) provide a detailed description of the incentives packages offered by each site. Gar- 
denhire-Crooks (2004) explores the sites’ experiences in implementing these plans and residents’ responses to 
them. 

14 Actual rent savings that a working family would enjoy under Jobs-Plus relative to traditional rent policies 
varied according to a number of factors. For example, in Dayton, a single parent with two children working full 
time at $6 per hour could see her rent drop by 50 percent or more relative to what she would pay under traditional 
rent rules, for a savings of nearly $1,700 to $2,300 per year (depending on the stage of the Jobs-Plus rent plan). If 
the same tenant earned $10 per hour, her rent savings under Jobs-Plus would amount to $3,600 to $4,200 per year. 
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earnings but with staged increases in the flat rate over time, and (2) reductions in the percentage 
of income paid in rent, to a rate that was lower than the traditional 30 percent. As further in- 
ducements to encourage residents to remain steadily employed, several sites allowed residents 
to earn credits toward a free month’ s rent by remaining employed for a specified period of time 
or to have some of their rent payments diverted into a special savings account that they could 
later redeem if they worked continuously. 

In addition to these rent-based work incentives, the sites were expected to educate resi- 
dents about the other financial work supports that are more generally available to low-income 
working families and individuals through a variety of government-funded programs (such as 
earnings disregards available under Temporary Assistance for Needy Families [TANF], child 
care subsidies, the EITC, and the Child Tax Credit). Although Jobs-Plus programs did make 
some attempts to advertise the EITC, they typically gave little attention to other financial work 
supports and, instead, concentrated most of their incentives marketing efforts on the rent incen- 
tives. They also gave little systematic attention to educating residents on how certain job choices 
— for example, a job offering a particular wage rate or number of hours of work per week — 
would affect their net family income, after taking into consideration work-related costs and the 
potential loss or reduction of other benefits, such as TANF or food stamps and the potential gain 
from the EITC. Nor did the staff regularly try to connect this information to the career guidance 
that they offered residents or to their recommendations concerning education or training op- 
tions. 15 In this sense, the services and incentives components of Jobs-Plus were not as well inte- 
grated as they might have been. 

Resident-to-Resident Outreach: The Centerpiece of Community Support 
for Work 

The third main component of Jobs-Plus was an explicit attempt to increase “community 
support for work.” Because this concept was imprecise and meant different things to different 
people, it was no surprise that sites generally chose to implement the other two program com- 
ponents first. 16 

The original conception of community support for work envisioned a wide range of ef- 
forts to infuse residents’ social networks with information, advice, and mutual support that 
would promote and facilitate work. 17 Part of this involved developing a cadre of residents who 
would function, in essence, as extension agents of the Jobs-Plus program through their day-to- 

15 See Gardenhire-Crooks, 2004. 

16 For a more detailed examination of the implementation of this component, see Kato (2004). 

n Riccio, 1999. This vision of building social capital in ways that focused on work was inspired by studies 
that underscore the role of social networks in the job-finding process. See, for example, Briggs, 1998; and Jar- 
gowsky, 1997. 
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day interactions with their neighbors in the development, communicating with them about work 
and training opportunities. Other ideas involved helping residents establish cooperative child 
care arrangements; building relationships between residents and professional working people 
living outside public housing who could become mentors or additional sources of information 
about good job opportunities; and building ties with local churches and other institutions in the 
larger community that might support the program’s work goals in a variety of ways. Also envi- 
sioned were changes in certain institutional practices that could support work, such as extending 
the hours that the housing management offices and the Jobs-Plus offices remained open so that 
employed residents need not miss work in order to attend to issues pertaining to their lease and 
rent or to get career counseling or other assistance from Jobs-Plus staff. 

Although many of these kinds of activities were tried at some points during the demon- 
stration (even within the year of program startup), most did not take root. What did take root — 
and grow — was the idea of using a small group of residents as extension agents of Jobs-Plus, 
and this became the centerpiece of the sites’ community support for work component. With the 
exception of Chattanooga, all of the Jobs-Plus programs established a network of “community 
coaches” — residents who, on a “neighbor-to-neighbor” basis and working in partnership with 
the Jobs-Plus staff, disseminated information about job openings and about the opportunity to 
get employment help from Jobs-Plus . 18 This often involved going door-to-door to distribute fly- 
ers and to talk with other residents about specific job openings, the rent incentives, education 
and training opportunities, and other services and activities available through the program. The 
coaches were paid a small stipend for their volunteer time, and they were overseen by the Jobs- 
Plus staff. The community coaches would also answer residents’ questions about Jobs-Plus and 
relay residents’ concerns to program staff. 

As one way to focus the community coaches’ efforts specifically on employment- 
related activities, the sites were encouraged to adopt a new job-related outreach “campaign” 
each month — for example, on promoting rent incentives, on-site employer recruitments, or 
information about the EITC. This work supplemented other efforts at the developments, includ- 
ing community wide events such as job fairs, health fairs, and holiday and back-to-school events 
that all sites sponsored as a way of fostering a stronger sense of community among residents. At 
the Los Angeles development — where this component was strongest — the community 
coaches spearheaded efforts to organize basic education classes at the development, and they 
recruited other residents to participate in those classes and provided child care for those who 
needed it while the classes were in session. 



l8 The residents who were recruited and trained for this purpose were known by different titles at each site: 
court captains (Baltimore), building captains (Dayton), community coaches (Los Angeles), community outreach 
workers (St. Paul), and resident outreach and orientation specialists (Seattle). 
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Sometimes the community coaches approached their role in a fairly perfunctory man- 
ner, simply dropping off flyers without engaging other residents. Ensuring that the coaches were 
a vigorous, promotional arm of the program was thus a constant challenge. It required careful 
selection and training of the individuals who would play this role and ongoing direction and 
oversight from the Jobs-Plus staff. In general, according to field research interviews with staff 
and residents, the community support for work component was a helpful strategy for “spreading 
the word” about Jobs-Plus. Having information come not just from professionals but from 
neighbors as well may have enhanced the legitimacy of the program in the eyes of the broader 
tenant community that often had reasons to be suspicious of new programs. 

Informal, On-Site Assistance: A Core Feature of Jobs-Plus 

Jobs-Plus’s case managers, job developers, and administrative staff coordinated out- 
reach activities, conducted intake and assessment, and offered job search assistance and case 
management services at their offices located at the housing developments. However, being on- 
site also created many informal opportunities for staff to leave their offices and go out to the 
homes, courtyards, and neighborhood comers and shops where residents were likely to be found 
— to get to know them, hear directly about their concerns and needs, and assist them in ad hoc, 
individualized ways. “We discovered [that] we needed to do training services in nontraditional 
ways,” said a staff member in Seattle. “We weren't taking advantage of our proximity if we 
didn’t cross lines, going to homes, babysitting, visiting families.” For instance, home visits dur- 
ing or after regular program hours permitted the staff to learn about people’s lives and “see 
things happening in the family.” Such visits helped break down cultural barriers with foreign- 
bom residents and encouraged those who suffered from serious family or mental health issues 
to consider referrals for professional help. Indeed, staff members emphasized the necessity of 
being “opportunistic” in taking advantage of every interaction in the development to assist resi- 
dents and build trust and credibility. In Dayton, a staff member talked about going outside for a 
cigarette as an occasion to grab residents as they went to and from work: “If I see somebody, if 
I’m hanging out in the back ... I holler.” Similarly, a staff member in Baltimore observed that, 
in meeting residents in the courtyards, “We can’t get from here to there without someone [ap- 
proaching us], and you have to deal. You have to satisfy that person's needs right where they 
are, or you say, ‘Come on and walk me down to the office’ or ‘Walk me to my car.’ And they 
will do that. We have some clients that will not come into the office .” 19 

The Jobs-Plus programs also sponsored various activities for the entire development, 
often in partnership with existing resident-led organizations. Usually these events were held on 
weekends, and they included picnics, holiday celebrations, basketball tournaments, job fairs, 



19 Kato, 2003a, p. 44. 
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health fairs, and so on. These events became a popular way to inform residents about program 
services and opportunities for employment. 

Collectively, these efforts helped staff develop a broad, holistic understanding of the 
needs and circumstances that could affect residents’ employment — including family problems 
and supports, social networks, and neighborhood conditions — which expanded their notion of 
services for employment. For instance, staff members in St. Paul and Seattle saw that problems 
with health, children and teens, immigration status, and finances could undercut a resident’s 
ability to secure and retain a job. Consequently, their notion of employment assistance included 
such efforts as accompanying residents to doctor’s appointments or immigration hearings and 
helping them to buy furniture or translate letters and bills, which could build trust that could 
lead to better engagement with the program. 



Residents’ Participation in Jobs-Plus 

Two Cohorts, Varying Experiences 

All the efforts described in the foregoing section helped Jobs-Plus become widely known 
among residents as a place to get help with employment needs. But how many residents took ad- 
vantage of what Jobs-Plus offered? To answer this question, measures of resident engagement in 
the program have been calculated using two overlapping samples of tenants: those residents of the 
Jobs-Plus developments who are part of the main impact analysis sample, referred to as the “1998 
cohort,” and those who are part of a supplementary impact sample, referred to as the “2000 co- 
hort.” The earlier cohort, as described in Chapter 2, is made up of all working-age, nondisabled 
residents aged 21 to 61 who were living in a Jobs-Plus development in 1998, the year when Jobs- 
Plus operations began in all sites. The experiences of this cohort thus reflect the behavior and out- 
comes of residents who had an opportunity to be involved with Jobs-Plus /rom its earliest days. 
However, the 1998 cohort also includes many residents who moved out of the Jobs-Plus devel- 
opments before the crucial rent incentives component became available (in mid to late 2000 in 
most sites) and, in general, before the program reached a more mature stage of operations after the 
initial “rollout period.” 

Move-out rates in some developments were much higher than had been anticipated at 
the start of the demonstration. As Table 3.2 shows, nearly one-third (31 percent) of residents in 
the 1998 cohort for all Jobs-Plus developments combined had moved away within two years 
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The Jobs-Plus Demonstration 
Table 3.2 



Move-Out Rates for the 1998 Cohort of Residents of the Jobs-Plus Developments 



Percentage Moving Out Within: 


Development 


One Year 


Two Years 


Three Years 


Baltimore 


30 


41 


51 


Chattanooga 


26 


36 


52 


Dayton 


35 


48 


58 


Los Angeles 


9 


17 


22 


St. Paul 


14 


27 


38 


Seattle 


11 


18 


31 


All developments combined 


21 


31 


42 



SOURCE: MDRC calculations using data from housing authority tenant (50058) records. 



NOTES: The 1998 cohort includes all residents of each public housing development in October 1998 who 
were between 21 and 61 years old and not listed as disabled on public housing authority records. 

The follow-up period for calculating the move-out rates began in October 1998. 



(that is, before October 2000), and 42 percent had done so within three years (before October 
200 1). 20 Move-out rates were highest in Baltimore, Chattanooga, and Dayton, where public 
housing functioned as transitional housing or housing of last resort for a large proportion of 
families. In those developments, over half the 1998 cohort (from 51 percent to 58 percent) had 
moved out within three years, with a quarter or more leaving as early as within one year. As 
mentioned in Chapter 2, softer housing markets in the Baltimore and Dayton areas undoubtedly 
contributed to these higher move-out rates. In addition, concerns over safety and drug use in the 
surrounding community and at the three developments were important factors propelling some 
residents to leave if they could. 21 Finally, the availability of extra Section 8 rent vouchers aimed 
at TANF leavers who were entering work may have contributed to the high move-out rates in 
Baltimore and Chattanooga. 22 At the other three developments (Los Angeles, St. Paul, and Seat- 



20 For a comparison of these move -out rates and the rates for residents of the comparison developments, see 
Appendix Table B.7. 

_I A special analysis of residential mobility among residents of the Jobs-Plus developments found that be- 
ing a victim of crime or threatening behavior was correlated with the likelihood of subsequendy moving (see 
Verma, 2003). 

22 Kato, 2003b. 
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tie), move-out rates were considerably lower though hardly trivial, with 22 percent to 38 percent 
of residents leaving within three years. 23 

The combination of a slow implementation of Jobs-Plus and, in some sites, a high 
move-out rate means that many residents in the 1998 cohort were never exposed to a full and 
mature Jobs-Plus program, making it harder, of course, for Jobs-Plus to affect their earnings and 
employment (see Chapter 4). Moreover, participation measures based on that cohort’s experi- 
ences may underestimate what the response to Jobs-Plus would have been when it operated in a 
way that was closer to its designers’ original vision. The 2000 cohort helps to correct for this 
limitation. It includes residents living in the Jobs-Plus developments in 2000 — about the time 
that fully formed Jobs-Plus programs were finally in place. 24 

Substantial Engagement That Varied Over Time and by Location 

Records kept by Jobs-Plus offer one source of insight into residents’ uses of the pro- 
gram’s services and incentives and how those patterns varied across sites and cohorts. 25 Using 
those data, Table 3.3 presents findings on two broad measures: 

• The Jobs-Plus rent incentives take-up rate, which refers to the percentage of 
all targeted residents (not just working residents) who belonged to house- 
holds receiving Jobs-Plus rent incentives. 26 

• The Jobs-Plus attachment rate, which refers to the percentage of all targeted 
residents who were personally enrolled in Jobs-Plus or, even if not enrolled, 
who belonged to households receiving Jobs-Plus rent incentives. (Being “en- 
rolled” means that a resident signed up for Jobs-Plus, and it is a reasonable 
indicator of having received at least some case management or other assis- 



“ 3 ln Seattle, the move-out rate jumped by 21 percentage points between the third and fourth years (Appendix 
Table B.7) as relocations under HOPE VI began to increase. 

24 The 2000 cohort includes residents from the 1998 cohort who did not move away and any new tenants who 
moved into the development after 1998 and were living there in 2000. 

“’Enrollment data in all sites except Seattle were collected from each Jobs-Plus program through an on-site 
review of the case files that were maintained on each program participant; in Seattle, the data came from elec- 
tronic records kept by the program (see Kato, 2003a). Data on rent incentives were obtained through payment 
spreadsheets maintained by the programs and provided to MDRC (see Gardenhire-Crooks, 2004). Data collection 
for the enrollment indicator covers the period from the start of Jobs-Plus in early 1998 (the exact startup date var- 
ied by site) through June 2001. Data on the use of financial rent incentives cover the period ending in December 
2002. On average, the length of participation follow-up was shorter for residents of the 2000 cohort than it was for 
the 1998 cohort. 

“ 6 A household needed to have at least one working member in order to be eligible for the rent incentives. 
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The Jobs-Plus Demonstration 



Table 3.3 

Selected Measures of Jobs-Plus Participation for Targeted Residents 
in the 1998 and 2000 Cohorts 



Measure 


Baltimore 


Chattanooga 


Dayton 


Los 

Angeles 


St. Paul 


Seattle 


All 

Developments 

Combined 


Rent incentives receipt 

Lived in a household 
that received 
rent incentives 
by December 2002 (%) 
1998 cohort 


12 


28 


30 


61 


67 


42 


40 


2000 cohort 


19 


38 


60 


72 


77 


50 


53 


Attachment rate 

Enrolled in Jobs-Plus 
by June 2001 or 
received rent incentives 
by December 2002 (%) 
1998 cohort 


50 


50 


63 


66 


85 


58 


62 


2000 cohort 


61 


66 


96 


76 


88 


67 


76 


Sample size 


1998 cohort 


378 


340 


425 


524 


316 


425 


2,408 


2000 cohort 


340 


373 


321 


598 


319 


398 


2,349 



SOURCES: MDRC calculations using data from Jobs-Plus case files, Jobs-Plus rent incentives records, and 
housing authority tenant (50058) records. 

NOTE: In the average for all developments combined, the results for each housing development are weighted 
equally. 
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tance.) 27 The attachment rate is thus a fairly inclusive measure of formal in- 
volvement or connection to Jobs-Plus, because it counts as “attached” people 
who met either one of these two conditions. 

The results in Table 3.3 show that, for all developments combined, about 40 percent of 
all targeted residents in the 1998 cohort had lived in households receiving the Jobs-Plus rent 
incentives at any time within the subsequent four years (that is, by December 2002). As ex- 
pected, because many of these residents had moved before the incentives became available, the 
rates were higher for the 2000 cohort, climbing by 13 percentage points, to 53 percent. 28 

Differences across sites were substantial. Baltimore ranked distinctively low on this 
measure, with 12 percent of residents in the 1998 cohort living in a household that received the 
incentives. This rate increased to only 19 percent for Baltimore's 2000 cohort. Chattanooga had 
the second-lowest rates (28 percent and 38 percent, respectively, for the two cohorts). For the 
2000 cohort, only Baltimore and Chattanooga had take-up rates under 50 percent. At the other 
extreme, a substantial majority of residents in Los Angeles and St. Paul made use of rent incen- 
tives (72 percent and 77 percent, respectively, of the 2000 cohort). 

The differences in participation rates between the earlier and the later cohorts were most 
pronounced in Dayton. Here, the high move-out rate in combination with the late availability of 
the incentives limited their use by the 1998 cohort. However, with an aggressive marketing effort, 
the program effectively reached many members of the 2000 cohort — residents who were living 
in the development when the rent incentives became available. For example, Table 3.3 shows that 
while only 30 percent of Dayton’s 1998 cohort had used the incentives, the rate for its 2000 cohort 
reached 60 percent — twice as high. The increase was less dramatic in the other sites. 

It might be thought that developments where the take-up rates were lower were ones 
that had a smaller proportion of households with an employed member — a key eligibility re- 
quirement to receive the rent incentives. But this was not the case. In fact, residents’ employ- 
ment rates did not differ widely across the sites. Moreover, when rates of incentives use were 
calculated only for targeted households in the 2000 cohort who were eligible for the rent incen- 



27 When residents showed up at the Jobs-Plus office to enroll, staff often wanted to take advantage of their 
presence there to begin an immediate discussion about their circumstances and job interests and how the program 
could help them. After they were enrolled, residents could be provided with job search and job placement assis- 
tance, refened to education or training activities, or assisted through such support services as bus tickets and help 
in locating or paying for child care. 

28 For more detailed analyses of the use of rent incentives, see Gardenhire-Crooks (2004). 
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tives (because they had at least one member employed), Baltimore still ranked lowest (23 per- 
cent), while Los Angeles and St. Paul ranked highest (75 percent and 84 percent, respectively). 29 

It appears that the very low incentives take-up rate in Baltimore and the higher rates in 
other sites resulted mostly from differences in the quality of administration and efforts to market 
the incentives to residents. As one illustration, the housing manager in the Los Angeles Jobs- 
Plus development personally signed off on every annual income and rent review, which was 
required of all public housing residents. Noting when individuals were candidates for Jobs-Plus, 
he called them in to the housing management office to talk about the program and encouraged 
them to enroll, sometimes even personally escorting them to the Jobs-Plus office. In St. Paul, 
housing authority staff took the lead in promoting and signing residents up for the incentives 
and, as in Dayton and Seattle, worked in close partnership with Jobs-Plus staff for this purpose. 
In sharp contrast, the housing authority in Baltimore had made processing resident applications 
for Jobs-Plus rent incentives a lower priority, and residents who did sign up often had to wait 
months before the adjustments to their rental bill took effect. Sometimes, after they began pay- 
ing a lower rent under the Jobs-Plus rules, the housing authority would mistakenly issue warn- 
ing notices that they had not paid their full rent on time. Not surprisingly, these actions bred 
cynicism about Jobs-Plus among other residents. Overall, the Jobs-Plus programs that most ag- 
gressively promoted the use of the incentives and that built strong partnerships for this effort 
with the housing management office generated a better response to the offer. 30 

The attachment rate — a broader measure that reflects either enrollment in Jobs-Plus or 
the use of its rent incentives or both — offers another way to gauge residents’ formal connec- 
tion to the program. This indicator shows that, across all developments, a majority of residents 
(62 percent of the 1998 cohort and 76 percent of the 2000 cohort) had a connection to Jobs-Plus 
— a significant accomplishment. 

An analysis of program case file records on Jobs-Plus enrollees from 1998 through 
2000 shows the extent to which residents participated in different types of employment-related 
activities (see Appendix Table B.8). 31 Across the developments, enrollees sought assistance 
with job referrals and job search more than other activities. In general, education and training 
activities (such as Adult Basic Education [ABE], English as a Second Language [ESL], and 



29 The rates by site for households in the 2000 cohort with employed members were as follows: Baltimore, 23 
percent; Dayton, 64 percent; Los Angeles, 75 percent; St. Paul, 84 percent; Seattle, 59 percent. The rates for simi- 
lar households in the 1998 cohort were somewhat lower: Baltimore, 17 percent; Dayton 33 percent; Los Angeles, 
67 percent; St. Paul, 73 percent; Seattle, 52 percent. Data limitations made it impossible to compute this measure 
for Chattanooga. 

30 Gardenhire-Crooks, 2004. 

3 'The samples used for collecting these participation data on enrollees are too small to show a breakdown by 
type of activity for the 1998 or the 2000 cohort. 
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vocational training courses) were more commonly sought in Los Angeles and St. Paul than in 
the other sites. 

The quantitative analysis of case file data looked only at recorded referrals to or partici- 
pation in various employment-related activities and did not examine help in accessing such sup- 
port services as child care or transportation assistance. Moreover, these data do not capture the 
common practice across the sites of staff offering work-related counseling or guidance when 
residents called or “dropped in” to the Jobs-Plus office to check in with staff, pick up bus to- 
kens, or use the fax or copy machines. Qualitative data from staff interviews and case file narra- 
tives suggest that these informal encounters often led to an extensive conversation about child 
care issues, a troublesome coworker, or a difficult commute — in other words, they became 
opportunities for the program to offer substantive though unscheduled help with employment- 
related issues and barriers. 

An alternative source of evidence on residents’ involvement in Jobs-Plus comes from 
the cross-sectional 2003 follow-up survey of residents, which is discussed more fully near the 
end of this chapter. As part of this survey, a sample of heads of households living in the Jobs- 
Plus developments in 2003 was asked a series of questions to determine whether Jobs-Plus had 
ever helped those residents participate in specific employment-related activities or obtain 
needed social services or ancillary services (such as transportation payments, child care assis- 
tance, or payments for books, tools, uniforms, and so on). Across the five survey sites (Seattle is 
not included), 54 percent of respondents — a substantial proportion for a voluntary program — 
said that they had been assisted by Jobs-Plus in these ways. Notably, this estimate does not take 
into account still other residents who were working and using the Jobs-Plus rent incentives 
without receiving other forms of assistance. 32 

Taken together, the evidence from the program case file records, the resident survey, 
and the qualitative field research suggests that Jobs-Plus had indeed achieved considerable 
reach into the tenant population of the participating public housing developments. And it did so 
across developments that differed markedly in terms of crime and safety, residents’ racial and 
ethnic backgrounds, residents’ family structures, and other conditions. While the findings do not 
necessarily mean that the program intensively served most residents and cannot speak to the 
quality of the services provided, 33 they make clear that Jobs-Plus “touched” in some way — 



32 By site, the percentages of respondents who said that Jobs-Plus helped them in these ways were as follows: 
Baltimore, 45 percent; Chattanooga, 60 percent; Dayton, 53 percent; Los Angeles, 38 percent; St. Paul, 67 percent. 

33 Data on the duration of receipt of rent incentives indicate that many residents who took up this benefit used 
it for an extended period of time. For example, across the sites, 68 percent of the recipients used the incentives for 
at least a year, and 32 percent used them for at least two years (Gardenhire-Crooks, 2004). These rates would have 
been even higher if the incentives had been available sooner and if the data collected for this part of the analysis 
had extended beyond 2002. 



56 




through concrete services, rent incentives, and formal and informal counseling and support — a 
majority of the residents it aimed to help and that it did so to a considerably higher degree 
among residents of the 2000 cohort, who were still living in the developments when Jobs-Plus 
was reaching its stride. 



Services and Incentives for Residents of the Comparison 
Developments 

Jobs-Plus far exceeded in scope and intensity the kinds of housing-authority-sponsored 
self-sufficiency services normally available on-site to residents of the demonstration’s comparison 
developments. At the same time, those residents were hardly an unserved group. Indeed, many of 
them — more than expected — received employment-related assistance of some kind. Part of this 
assistance came through housing authority programs, but much more of it was obtained through 
programs operating outside public housing. Residents also had access to some rent-based work 
incentives that were available authority wide under the 1998 federal housing legislation (the Qual- 
ity Housing and Work Responsibility Act, or QHWRA) and under earlier reforms. 

Less Intensive On-Site Assistance and Outreach 

All the housing authorities offered residents of the comparison developments at least 
some self-sufficiency programs other than Jobs-Plus (see Appendix Tables B.l through B.6). 
For example, some comparison developments operated on-site basic education or computer 
classes. Some had after-school and tutoring programs for children and youth. A few had on-site 
community centers, health clinics, and job readiness classes. Some provided access to social 
workers or even employment counselors at the developments, although such staff were usually 
not stationed full time at any one development, as were Jobs-Plus staff. The housing authorities 
also offered a limited number of opportunities for apprenticeship training, such as in buildings 
maintenance and groundskeeping. 

As mentioned above, residents of the comparison developments also had access to rent- 
based work incentives that were available authority wide. However, these incentives were less 
generous than those offered by Jobs-Plus or were available only to residents who met certain 
conditions. For example, the flat rents introduced under QHWRA were based on the market 
value of residents’ apartments, making the rents considerably higher than the flat rents available 
under Jobs-Plus. Other rent policies under QHWRA called for disregarding increases in resi- 
dents’ income due to earnings when their rents were calculated. However, this provision applied 
only to particular categories of working residents, such as those who had received TANF bene- 
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fits within the prior six months or who had not worked for at least a year. 34 Not only were nor- 
mally available rent incentives less generous or less broadly targeted, but housing authorities 
were also much less aggressive in marketing their incentives to residents than were the Jobs- 
Plus programs. 

Although the comparison developments generally offered much less intensive employ- 
ment assistance to their residents than was available to residents of the Jobs-Plus developments, 
two important qualifications to this conclusion should be noted. In St. Paul, one of the two com- 
parison developments experienced an unanticipated growth in on-site services after the demon- 
stration began — to a level that rendered that development problematic as a benchmark for assess- 
ing the effectiveness of Jobs-Plus. Simply put, the development's program looked too much like 
Jobs-Plus to permit a fair comparison with Jobs-Plus. Consequently, that development plays a 
more limited role in the impact analysis presented in Chapters 4 and 5. 35 For different reasons, 
Seattle’s single comparison development also saw a gradual expansion of on-site services, and 
these were accompanied by the authority wide adoption of rent-based work incentives that rivaled 
those of Jobs-Plus. 36 Nonetheless, this development remained a viable comparison development, 
especially during the first few years of the follow-up period. However, the degree of the service 
contrast experienced by its residents and those in Jobs-Plus lessened over time, particularly as 
HOPE VI relocation and demolition advanced at the Seattle Jobs-Plus development. 37 



34 Under this provision, 100 percent of the earnings increase due to employment was disregarded in calculat- 
ing a resident’s rent for 12 months. After this 12-month period, rent for the next 12 months would increase by 
only half the amount it would have been raised under the traditional rules. Alternatively, the housing authority 
could have residents pay the higher rent and deposit the difference into an escrow account. For a fuller discussion 
of the QHWRA rent reforms, see Miller and Riccio (2002) and Sard and Bogdon (2003). 

In St. Paul — in response to increasingly urgent needs of the tenant population at this development 
(McDonough Homes) — the housing authority and other local partners increasingly invested in on-site social 
services and employment counseling over the course of the Jobs-Plus demonstration. Field research conducted at 
that comparison development showed that these activities far and away exceeded the efforts observed at the com- 
parison developments in the other cities and at the second comparison development in St. Paul (see Appendix 
Table B.5). Because the service environment at McDonough Homes appears to have been so similar to that of 
Jobs-Plus, this development is only included as part of sensitivity tests of the impact findings (see Appendix D). 

36 From about 2000 onward, the Seattle Housing Authority, taking advantage of special funding through 
the U.S. Department of Labor’s Welfare-to-Work program and other resources, and in cooperation with other 
local service partners, increasingly made employment assistance available to residents at the comparison de- 
velopment (Yesler Terrace). More important, perhaps, is that it used special waiver authority obtained under 
HUD’s Moving to Work demonstration, which was intended to promote innovative management reforms and 
self-sufficiency initiatives, to introduce rent-based work incentives at Yesler Terrace (and other housing au- 
thority properties) that rivaled those of Jobs-Plus. At the same time, Yesler Terrace did not have the dedicated 
on-site staffing and intensive outreach efforts — which were so central in Jobs-Plus at Rainier Vista — to pro- 
mote its services and incentives. 

37 It should also be noted that conditions changed during the latter part of the follow-up period in the Los 
Angeles comparison development (Dana Strand Village). During the period of the demonstration, the Los An- 
geles housing authority received a HOPE VI grant to tear down and rebuild Dana Strand Village. The housing 

(continued) 
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Substantial Employment Assistance in the Larger Community 

The services available to residents of the comparison developments (and to those of the 
Jobs-Plus developments) were not limited to services provided through the housing authority. 
Residents could also obtain services through the local welfare department (which operated 
mandatory welfare-to-work programs); the workforce system (which funded a variety of pro- 
grams under the Workforce Investment Act); and a host of other human service agencies, 
schools, and training providers in the local community. Indeed, as discussed in Chapter 1, Jobs- 
Plus was implemented at a time of significant federal reform in these policy domains (see Fig- 
ure 1.2), and these reforms promised to increase the use of work-promoting assistance and in- 
ducements by many low-income people, including public housing residents. Although residents 
of the comparison developments did not have the advantage of Jobs-Plus’ s interagency partner- 
ships and intensive on-site assistance, neither were they insulated from local agencies’ own re- 
cruitment and marketing efforts or, for some residents, welfare-to-work participation require- 
ments. Consequently, in this environment, the capacity of Jobs-Plus to create a large difference 
in service receipt (or “service difference,” for short) between tenants of the Jobs-Plus and com- 
parison developments could not be taken for granted. 



Estimates of the Service Difference: Jobs-Plus Versus the 
Comparison Developments 

How big a service difference emerged? This question is not an easy one to answer with 
the data available to the evaluation. Ideally, longitudinal information on participation and ser- 
vice use would be available for residents in the main impact sample (the 1998 cohort) from the 
start of Jobs-Plus through the end of its operating period as part of the demonstration (generally, 
December 2003), regardless of whether those residents continued living in public housing. This 
information would need to come from a survey of residents in order to capture their use of any 
of a wide variety of services offered within or outside public housing. However, it was clear 
early on that tracking and interviewing residents who moved out of the developments would 
take far more resources than were available for this part of the research. 

As an alternative, a follow-up survey wave was administered in early to mid-2003 to a 
cross-section of working-age, nondisabled residents living in the Jobs-Plus or comparison devel- 
opments in five of the six sites. (The follow-up survey was not conducted in Seattle because of 
language barriers, the advent of the HOPE VI redevelopment process, and limited evaluation re- 
sources for that site after it was withdrawn from the national demonstration.) All respondents were 



authority began relocating some residents near the end of 2001. The process began slowly and proceeded in 
stages through several sections of the development. By December 2002, 34 percent of households had moved 
out of Dana Strand Village, and 59 percent had done so by April 2003. 
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household heads who had lived in the developments for at least a year prior to the interview. The 
latter condition was intended to ensure that respondents would have had a reasonable opportunity 
to be exposed to Jobs-Plus or any alternative programs offered by the housing authorities at the 
comparison developments. (See Appendix C for further details on the resident survey.) 

Although the results do not fully correspond to the experiences of residents in the 1998 
cohort (especially in the high-mobility sites), the 2003 survey offers the best evidence available 
on whether Jobs-Plus boosted service receipt among the residents of the program developments 
relative to those in the comparison developments. 38 The overall findings for all the sites com- 
bined (excluding Seattle) are presented in Table 3.4. 

The experiences of residents of the comparison developments offer a benchmark for as- 
sessing the added contributions of Jobs-Plus by indicating what the levels of service receipt for 
the Jobs- Plus group would have been in the absence of the program. The results show that these 
levels would have been substantial. For example, the top row in Table 3.4 reveals that 46 per- 
cent of survey respondents in the comparison developments reported that they had participated 
in an employment-related activity (such as job search assistance or education or training) 
through any program or agency within the 12 months prior to the survey interview. However, 
as the table’s next row shows, only 13 percent of those residents participated in such activities 
with the help of the housing authority or a program at their development. 

On the first of these measures — participation in activities through any program or 
agency — the rate for survey respondents in the Jobs-Plus developments was about 58 per- 
cent, reflecting a modest though statistically significant increase of 12 percentage points over 
the rate (46 percent) observed for their counterparts in the comparison developments. How- 
ever, the difference on the second measure was larger: Respondents in the Jobs-Plus devel- 
opments (taking advantage of the on-site Jobs-Plus program) were more likely — by a statis- 
tically significant 1 7 percentage points — than those in the comparison developments to have 
participated in such activities with help from the housing authority or a program at their hous- 
ing development. In effect, Jobs-Plus, as a place-based initiative, appears to have had a bigger 
influence on how residents got employment-related activities than on residents’ overall likeli- 
hood of getting such services. 

Table 3.4 also distinguishes participation in activities focused on finding a job (such as 
individual or group job search assistance) from participation in education and training. In gen- 
eral, respondents in the Jobs-Plus developments were likely to have participated in each of these 

38 To help control for differences between the program and comparison developments in the types of people 
responding to the survey, the differences in participation and other outcomes were regression-adjusted. The fol- 
lowing background characteristics of the sample members were statistically controlled: average number of years 
living in the development, presence of more than one adult in the household, race or ethnicity, gender, and age. 
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Table 3.4 

2003 Follow-Up Survey (Household Heads) 

Use of Services and Incentives by Residents of the Jobs-Plus Developments and 
Their Comparison Developments, Five Sites Combined (Excluding Seattle) 



Measure (%) 


Jobs-Plus 

Developments 


Comparison 

Developments 


Difference 


Participation and service use 

Participated in any employment-related activity 
within prior 12 months with help from: 

Any program/agency 


57.6 


45.5 


12.1 *** 


Housing authority or any program at development 


30.5 


13.4 


17 \ *** 


Within prior 12 months, participated in: 
Any job search activity 


42.1 


27.9 


14 2 *** 


Any education or training activity 


42.7 


38.1 


4.6 * 


Received any ancillary or social services 
within prior 12 months with help from: 
Any program/agency 


50.5 


42.3 


g 2 *** 


Housing authority or any program at development 


29.0 


10.1 


lg 9 *** 


Financial incentives 

Heard of rent-based work incentives 


65.5 


41.2 


24 2 *** 


Currently using rent-based work incentives 


45.5 


23.7 


21.8 *** 


Currently using rent-based work incentives ( among 
currently working residents only ) a 


53.8 


32.2 


NA 


Heard of EITC b 


60.8 


54.3 


6.5 ** 


Was encouraged to use EITC by housing 
authority or any program at development 


11.8 


6.0 


5.8 *** 


Used EITC during prior year 


33.3 


26.9 


g 4 *** 


Services and incentives 

Participated in any employment-related activities within prior 

12 months or currently using rent-based work 

incentives 


61.4 


43.9 


17 5 *** 


Overall current self-sufficiencv efforts 

Currently participating in any work-related activity, currently 
working, or looked for work within the past four weeks 


80.1 


72.4 


7 g *** 


Sample size 


668 


808 





(continued) 
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Table 3.4 (continued) 



SOURCE: MDRC calculations using data from the 2003 follow-up survey. 

NOTES: Estimates of program-comparison development differences control for various background 
characteristics of respondents. 

Statistical significance levels are indicated as *** = 1 percent; ** = 5 percent; and * = 10 percent. 
“Because the calculations for this measure exclude respondents who were not working at the time of the 
interview, rather than all respondents, no differences in take-up rates between the Jobs-Plus and comparison 
developments are computed. 

b Respondents in St. Paul were not asked this survey question. 



categories of activities at higher rates than did their comparison development counterparts. 
However, the differential was larger for job search assistance (14 percentage points) than for 
education and training (about 5 percentage points). 

Jobs-Plus also offered various forms of ancillary services that could help residents par- 
ticipate in activities or in work (such as help paying for books, fees, or tools) and assistance ar- 
ranging for important social services (such as counseling for family problems). Survey respon- 
dents in the Jobs-Plus developments were more likely than those in the comparison develop- 
ments to use such services. Again, the difference was larger for services obtained with the help 
of the housing authority or a program at the development (19 percentage points) than for such 
services received from any source (8 percentage points). 

Survey respondents in the Jobs-Plus developments were also nearly 25 percentage 
points more likely than respondents in the comparison developments (about 66 percent versus 
41 percent) to have heard about rent-based work incentives. They were also more likely, by 22 
percentage points, to report using these incentives at the time of the interview (46 percent versus 
24 percent). As previously noted, Jobs-Plus aggressively marketed these incentives to residents. 
In contrast, the housing authorities did much less to promote the more limited incentives avail- 
able to residents of the comparison developments under the 1998 federal housing legislation. 
Respondents in the Jobs-Plus developments were more likely than those in the comparison de- 
velopments to have heard of the EITC and to have used it in the prior year, although the differ- 
ences were small (about 6 percentage points on each measure). 

Finally, a more summary measure was constructed to assess the proportion of respon- 
dents who used at least one of the two main categories of work-promoting assistance that Jobs- 
Plus offered. It captures participation in employment-related activities within the past 12 months 
or use of rent incentives at the time of the follow-up survey interview. On this combined meas- 
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ure, respondents in the Jobs-Plus developments again ranked higher than their comparison de- 
velopment counterparts, by 18 percentage points (61 percent versus 44 percent). 39 

In sum, although the 2003 survey data must be interpreted cautiously, these results sug- 
gest that Jobs-Plus increased residents’ likelihood of using employment-related services; more 
important, it changed how they got those services. Jobs-Plus also appears to have increased the 
extent to which residents used rent-based work incentives, relative to what would have occurred 
in the absence of the program. Although none of these differences are exceptionally large, they 
are in the direction expected according to the theory underlying Jobs-Plus, as described in 
Chapter 1. A similar overall pattern of service differences between residents of the Jobs-Plus 
and comparison developments is also evident, to varying degrees, in each of the five survey 
sites, as Appendix Table B.9 illustrates. 40 



Summary and Conclusions 

The Jobs-Plus model presented the local collaboratives with the challenge of imple- 
menting as comprehensive and ambitious an employment initiative as has ever been attempted 
for public housing. Many problems were encountered, such as building the collaboratives them- 
selves into functioning entities and getting them to make local program design decisions in a 
timely way, arranging for appropriate staffing for the program, sustaining a funding base, sus- 
taining support among senior housing authority officials, operating all three program compo- 
nents together, and dealing with diverse tenant populations that had significant employment bar- 
riers and faced other impediments to sustained employment. 

At the same time, much was accomplished: The collaboratives in at least four of the six 
sites were able to build coherent and sustainable programs that were operated at a reasonable 
level of quality, even if all three components of the model were not well integrated. The sites 
made good progress in “saturating” the developments with information about Jobs-Plus and the 
work-related opportunities it offered to residents. And although the program was entirely volun- 
tary, a majority of targeted residents came forward seeking to take advantage of what it offered 
them. It is true that even the better-performing sites had room to improve their operation of the 
program. But, on the whole, their experiences demonstrate that the Jobs-Plus model is feasible 
to operate in very different kinds of local settings and that it can hold a powerful appeal to a 

39 Finally, an additional analysis of the survey participation measures was conducted in the three better-run 
sites (Dayton, Los Angeles, and St. Paul), where welfare administrative records data allowed the survey sample 
to be split between household heads who had received welfare (TANF) payments in the prior year and those 
who had not. The results suggest that, for both subgroups, Jobs-Plus ratcheted up their use of rent-based finan- 
cial incentives, participation in employment-related activities, and the use of ancillary and other support ser- 
vices (see Appendix Tables B. 10 and B. 1 1). 

40 Because of the small survey sample sizes for each site, extra caution is urged in comparing results across sites. 
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large proportion of a housing development's working-age residents — regardless of the racial or 
ethnic composition of the tenant population. 

The variation in the quality of Jobs-Plus implementation across sites is important to un- 
derscore in advance of the impact findings discussed in Chapters 4 and 5. Of course, it is diffi- 
cult to gauge overall implementation quality with any precision and to offer a clear ranking of 
the sites' programs. Part of the difficulty is that the yardsticks forjudging performance are not 
clear-cut, and the programs were not necessarily equally strong — or weak — across all aspects 
of performance. The quality of implementation also varied over time. For example, the Balti- 
more program made a promising start; by 2000, however, it suffered a steady decline, with a 
loss of staff, retrenchment of services offered by local agency partners, and poor housing au- 
thority implementation of the rent incentives. In contrast, the Los Angeles program had great 
difficulty finding its footing early on and nearly collapsed, but its chronic staffing problems 
were finally resolved by 2000, and it evolved into one of the strongest Jobs-Plus programs. Also 
noteworthy is that St. Paul and Seattle were the sites quickest to implement the rent incentives, 
although, later in the follow-up period, Seattle — with the onset of HOPE VI relocations and 
demolition — stopped allowing new enrollments in the rent incentives program and essentially 
stopped assisting residents who were relocated to homes outside the Jobs-Plus development. 

With these complicated patterns in mind, the fairest summary assessment is as shown in 
Table 3.5. During the period from 2000 to 2003 (after rent incentives had become available eve- 
rywhere), overall implementation of Jobs-Plus was strongest in three sites: Dayton, Los Ange- 
les, and St. Paul. Although these sites had room to improve, they came closest to implementing 



The Jobs-Plus Demonstration 
Table 3.5 

Overall Ranking of Sites’ Implementation of Jobs-Plus 



Ranking 


Site 


Stronger 


Dayton, Los Angeles, St. Paul 


Problematic 


Baltimore and Chattanooga 


A HOPE VI site 


Seattle 


(special case) 
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the Jobs-Plus model as the demonstration’s designers had envisioned it. Implementation was 
most problematic in Baltimore and Chattanooga, to an important extent because of diminished 
attention from the housing authority in each of those sites. Seattle is a special case. While it had 
operated a strong program during the first few years, the advent of its HOPE VI redevelopment 
process made the context within which Jobs-Plus was implemented unique among the sites. 
Over time, this shifted the focus of the program, broadening the target group to include the eld- 
erly and the disabled, expanding social services, and lessening the intensity of outreach and em- 
ployment assistance to the original Jobs-Plus target group. It most importantly disrupted the 
program by displacing tenants and tearing down the development. 

Given the substantial cross-site differences in operating Jobs-Plus, the impact analysis 
presented in Chapters 4 and 5 examines the program’ s effectiveness using the sites in three dif- 
ferent ways: (1) with all six sites combined, (2) with the three “stronger” implementation sites 
combined, and (3) separately for each site. Examining the program’s effects from these different 
vantage points sets the stage for drawing inferences about the connections between the sites’ 
implementation experiences described in this chapter and the effects of Jobs-Plus on residents’ 
labor market and welfare outcomes. 
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Chapter 4 



Work and Welfare Impacts 
on Public Housing Residents 

Chapters 4 and 5 examine the impacts of Jobs-Plus on public housing residents’ work 
and welfare — that is, the changes that it caused in these outcomes for the program group rela- 
tive to the outcomes for the comparison group, which did not have access to Jobs-Plus. This 
chapter focuses on Jobs-Plus’ s effects for persons who were residents of public housing when 
the program was launched (1998), regardless of where they lived subsequently and where they 
had lived previously. This is the program’s effect on that specific cohort of people. Chapter 5 
focuses on how Jobs-Plus affected the quarter-by-quarter levels of work and welfare outcomes 
in the public housing developments where the program was run, regardless of who lived in 
these developments before and after it was launched. Ignoring the outcomes of people who 
move, this is the program’ s effect on place. As noted earlier, because residents move into and 
out of public housing in large numbers, it was necessary — in order to tell a balanced story — 
to conduct the analysis of program effects from the perspectives of both people and place. 

Specifically, this chapter addresses the following questions: 

• By how much did Jobs-Plus increase the average earnings of public housing 
residents and the percentage of residents who were employed? 1 

• How did these effects vary across the six years in the follow-up period for the 
Jobs-Plus evaluation, the six sites participating in the Jobs-Plus demonstra- 
tion, earlier versus later cohorts of public housing residents, and different 
subgroups of residents? 

• By how much did Jobs-Plus reduce average welfare benefits received by 
public housing residents and the percentage of residents who received those 
benefits? 

Findings indicate that, at four of the six study sites (where implementation of the pro- 
gram was strongest), Jobs-Plus markedly increased the earnings of public housing residents 
(with positive but less clear effects on employment rates) once the program was in place. These 
impacts are especially impressive, given that they persisted even through the onset of a national 
economic recession and that they represent “value added” by the program over and above any 



'This study focuses on earnings and employment in the formal labor market. It does not examine resi- 
dents’ participation in informal work activities that might provide material resources but are not recorded. 
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effects produced by recent national reforms to social support systems (welfare reform, public 
housing reform, and changes in the workforce development system). At three of those four sites, 
program impacts were sustained for at least four years and showed no signs of diminishing. 
These findings held for two different cohorts of public housing residents and for numerous sub- 
groups of residents. At the fourth site, the effects of Jobs-Plus disappeared with the onset of a 
HOPE VI demolition and renovation program that displaced large numbers of residents. At the 
remaining two sites, where program implementation was problematic, Jobs-Plus had no effects 
on residents' earnings. 

Even though Jobs-Plus was not a welfare-to-work program, assessing its effects on wel- 
fare receipt was judged important because so many residents of public housing were welfare 
recipients when the initiative was planned. The study’s findings suggest that although welfare 
receipt by residents dropped precipitously after Jobs-Plus was launched, this decline was no 
greater than what would have occurred without the program. The decline is most likely due to 
forces — such as the booming economy, welfare reform, and increases in the generosity of the 
Earned Income Tax Credit (EITC) — that are viewed by many as causing the dramatic declines 
in welfare rolls that occurred throughout the United States at the time. 



Measuring Program Impacts 

At each study site, one Jobs-Plus development and one or two comparison develop- 
ments were randomly selected (through a lottery) from a matched pair or triplet of eligible pub- 
lic housing developments nominated by the local public housing authority. Housing authority 
records were then used to identify all able-bodied working-age adults (defined as persons who 
were 21 to 61 years old and not listed as disabled) who were residents of the Jobs-Plus devel- 
opments or the comparison developments when the demonstration was launched in October 
1998. This group is referred to as the “1998 cohort.” Cohort members from the Jobs-Plus de- 
velopments are referred to as “the Jobs-Plus group,” and those from comparison developments 
are referred to as “the comparison group.” 

Data on total quarterly earnings were obtained for each 1998 cohort member from wage 
records of state Unemployment Insurance (UI) agencies. These data are used widely to evaluate 
welfare and employment programs and represent over 90 percent of the jobs in the formal labor 
market. 2 UI wage records were obtained for up to six years before Jobs-Plus was launched (its 
baseline period) and six years after Jobs-Plus was launched (its follow-up period), regardless of 
where cohort members lived at the time. From these data, two measures were created for each 
Jobs-Plus group and comparison group. One measure, average quarterly earnings, is expressed 



"Komfeld and Bloom, 1999. 
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in 2003 dollars and includes values of zero for persons who were not employed. The second 
measure, quarterly employment rates, is defined as the percentage of persons with any UI- 
reported earnings in a quarter. 

Monthly data on total welfare benefits (from Aid to Families with Dependent Children 
[AFDC] or Temporary Assistance for Needy Families [TANF]) that were paid to members of the 
1998 cohort were obtained for three sites — Dayton, Los Angeles, and St. Paul — from the ad- 
ministrative records of state or county welfare agencies. 3 These data, which cover the Jobs-Plus 
baseline period and follow-up period, report total benefits paid to the welfare case to which each 
person was attached during a month (if he or she was attached to a case). From this information, 
two quarterly measures were created for every Jobs-Plus group and comparison group. One 
measure, average quarterly benefits received, is expressed in 2003 dollars and includes values of 
zero for persons who did not receive benefits. The second measure, quarterly benefit receipt rates, 
is defined as the percentage of persons who received at least one dollar of benefits in a quarter. 

The impact of Jobs-Plus on work and welfare was estimated by comparing changes in 
outcomes after Jobs-Plus was launched for the Jobs-Plus group with changes in outcomes for 
the comparison group. Appendix D describes the analysis. 

Because many 1998 cohort members moved away during the two years that Jobs-Plus 
was being implemented, some were not exposed to the full program. To help account for this 
“dosage dilution,” the analysis was repeated for a later cohort of sample members defined as all 
working-age, nondisabled adults who were residents of a Jobs-Plus development or comparison 
development in October 2000 (“the 2000 cohort”). In addition, to assess the robustness of the 
program, the analysis was repeated for subgroups of the 1998 cohort defined in terms of indi- 
vidual characteristics for which data were available (gender, race/ethnicity, age, past employ- 
ment, past welfare receipt, past residential tenure, and future residential mobility). 



The Impact of Jobs-Plus on Residents’ Work 

This section examines the impact of Jobs-Plus on average earnings and employment 
rates for all the study sites combined, for the stronger implementation sites combined, for each 
site separately, for two cohorts of residents, and for several subgroups of residents. 



3 Welfare payments data were not received from Baltimore and Chattanooga. They were not requested 
from Seattle because of the site’s change in status due to HOPE VI. 
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Large Earnings Impacts and Smaller Employment Impacts for All the 
Sites Combined 

Although the full story of the effects of Jobs-Plus lies in its site-by-site details, it is help- 
ful to begin with a summary of the data combined across sites. Because the implementation 
success of Jobs-Plus varied so much across sites, Figure 4.1 combines the sites in two different 
ways. The graph in Panel A combines all the sites, which addresses the question “What is the 
average impact of Jobs-Plus for all versions of the program that are implemented?” The graph 
in Panel B combines the three sites that had stronger implementation, which addresses the ques- 
tion “What is the average impact of Jobs-Plus when it is implemented relatively well?” 

The graphs illustrate the time paths of average quarterly earnings with each site 
weighted equally. 4 Findings for the Jobs-Plus group are represented by a solid line, and those 
for the comparison group are represented by a dashed line. The baseline period in the figure is 
from 1994 through 1997 — the pre-program years for which all sites had data. This is followed 
by data for the two-year program rollout period (1998 and 1999) plus four more years after the 
program was implemented (2000 through 2003). 

Consider the findings for all sites combined (Panel A of Figure 4.1). First, note that 
mean quarterly earnings for the Jobs-Plus group and the comparison group roughly doubled 
during the four baseline years in the figure. When presented with more job opportunities pro- 
duced by a dynamic economy, public housing residents in the sample substantially increased 
their work in the formal labor market. Next, note that quarterly earnings for the Jobs-Plus group 
and the comparison group were virtually identical throughout the baseline period. Hence, these 
groups were well matched for a long time. 

While the program was being rolled out, earnings for the Jobs-Plus group and compari- 
son group continued to rise at the same rate. But once the program was implemented, the Jobs- 
Plus group began to pull ahead, and its lead was sustained for the rest of the follow-up period. 
Thus, on average, Jobs-Plus increased the earnings of public housing residents once the pro- 
gram was in place. 

Panel B of Figure 4.1 presents the results for the three stronger implementation sites 
(Dayton, Los Angeles, and St. Paul) combined. It illustrates the same rapidly rising baseline 
earnings levels and the close match for the two study groups, which continued throughout the 



4 Mean quarterly earnings and quarterly employment rates were computed for the combined Jobs-Plus 
group by summing the corresponding site-specific means or rates and dividing by the number of sites. Findings 
for the combined comparison group were computed similarly. For sites with two comparison developments, an 
equally weighted average was computed for these developments before the site’s comparison-group findings 
were averaged with those for the other sites. 
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The Jobs-Plus Demonstration 



Figure 4.1 



Pooled Average Quarterly Earnings for the 
Jobs-Plus Group and Its Comparison Group (1998 Cohort) 



A. All Sites Combined 




oooooooooooo 

◄ Baseline Follow-Up ► 



B. Stronger Implementation Sites Combined 




000000000000 
M Baseline Follow-Up ► 



SOURCES: MDRC calculations using data from housing authority tenant (50058) records and state 
Unemployment Insurance (UI) wage records. 

NOTE: The 1998 cohort includes all residents of a Jobs-Plus development or a comparison 
development in October 1998 who were between 21 and 61 years old and were not listed as disabled 
by their public housing authority. 
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rollout period. However, the subsequent earnings gains for the Jobs-Plus group in this graph are 
much larger than in Panel A. This indicates that when Jobs-Plus was implemented relatively 
well, its effects on earnings were much larger than average. 

Table 4. 1 presents the results of a statistical analysis of the quarterly earnings data in Fig- 
ure 4.1, showing findings for all sites combined and for the three stronger implementation sites. 
Consider the findings for all sites combined (Panel A). The first column indicates that average 
annual earnings for the Jobs-Plus group continued to rise, from $5,865 in 1998 to $8,649 in 2001, 
followed by a slight decline thereafter as the national economic recession set in. The second col- 
umn indicates that, relative to the comparison group, Jobs-Plus changed the average earnings of 
residents in all sites combined by -$4 and $180 during 1998 and 1999, when the program was 
being implemented. However, in following years, it increased residents’ average earnings relative 
to the comparison group by $461, $619, $440, and $472, for an average of $498 per year. 

The third column of Panel A in Table 4.1 presents estimates of what average earnings 
would have been for the Jobs-Plus group without the program. This equals the difference be- 
tween earnings that actually were observed for the Jobs-Plus group and the estimated program 
effect. For example, the $5,869 of estimated earnings without Jobs-Plus in 1998 equals the ob- 
served earnings of $5,865 minus the estimated program effect of -$4. This result provides a ba- 
sis for expressing program effects as a percentage of what earnings would have been without 
the program, which are referred to as “percentage changes” and are reported in the rightmost 
column. For example, the estimated $498 average annual program effect for the period 2000 to 
2003 represents a 6.2 percent gain relative to what earnings would have been without Jobs-Plus. 

Results for the stronger implementation sites in Panel B of Table 4. 1 indicate much lar- 
ger effects once the program was implemented. These estimates range from $7 14 (8.8 percent) 
in 2000 to $1,543 (19.5 percent) in 2003, for an average of $1,141 per year (14.1 percent) from 
2000 to 2003. This implies that Jobs-Plus increased total earnings per person by $4,564 over 
four years. Therefore, the results of the statistical analysis mirror those of the graphical analysis, 
with both indicating that the effects of Jobs-Plus on earnings were much greater than average in 
sites where the program was implemented relatively well. 

Figure 4.2 presents graphical analyses of quarterly employment rates. Results in Panel 
A for all sites combined illustrate that employment rates rose dramatically, from about 30 per- 
cent in 1994 to about 50 in 1998. This was the case for both the Jobs-Plus group and the com- 
parison group. Thus, prior to the launch of Jobs-Plus, the two groups were well matched in 
terms of employment rates. However, the rapid escalation of employment rates that occurred 
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The Jobs-Plus Demonstration 
Table 4.1 

Pooled Average Annual Outcomes for Earnings, 
by Site Combination and Follow-Up Period (1998 Cohort) 



Follow-Up Period 


Observed 
Outcome with 
Jobs-Plus 


Estimated 
Effect of 
Jobs-Plus 


Estimated 
Outcome Without 
Jobs-Plus 


Estimated 
Percentage 
Change 
in Outcome Due to 
Jobs-Plus 


A. All Sites Combined ($) 










1998 


5,865 


-4 


5,869 


-0.1 


1999 


7,379 


180 


7,199 


2.5 


2000 


8,329 


461 ** 


7,868 


5.9 


2001 


8,649 


619 *** 


8,030 


7.7 


2002 


8,629 


440 ** 


8,189 


5.4 


2003 


8,578 


472 ** 


8,106 


5.8 


2000-2003 


8,546 


49g *** 


8,048 


6.2 


B. Stronger Implementation Sites Combined ($) 








1998 


6,089 


173 


5,916 


2.9 


1999 


7,619 


209 


7,410 


2.8 


2000 


8,793 


714 ** 


8,079 


8.8 


2001 


9,256 


1,135 *** 


8,121 


14.0 


2002 


9,419 


1 i7i *** 


8,248 


14.2 


2003 


9,443 


1,543 *** 


7,900 


19.5 


2000-2003 


9,228 


1 i4i *** 


8,087 


14.1 



SOURCES: MDRC calculations using data from housing authority tenant (50058) records and state 
Unemployment Insurance (UI) wage records. 



NOTES: All findings are reported in 2003 dollars. 

The 1998 cohort includes all residents of a Jobs-Plus development in October 1998 who were between 
21 and 61 years old and were not listed as disabled by their public housing authority. 

Statistical significance levels are indicated as *** = 1 percent; ** = 5 percent; and * = 10 percent. 

There were 2,123 persons in the Jobs-Plus group and 2,651 persons in the comparison group for all sites 
combined. There were 1,088 persons in the Jobs-Plus group and 1,209 persons in the comparison group for 
the stronger implementation sites combined. 
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The Jobs-Plus Demonstration 



Figure 4.2 

Pooled Average Quarterly Employment Rates for the 
Jobs-Plus Group and Its Comparison Group (1998 Cohort) 



A. All Sites Combined 
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B. Stronger Implementation Sites Combined 
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SOURCES: MDRC calculations using data from housing authority tenant (50058) records and state 
Unemployment Insurance (UI) wage records. 

NOTE: The 1998 cohort includes all residents of a Jobs-Plus development or a comparison 
development in October 1998 who were between 21 and 61 years old and were not listed as disabled 
by their public housing authority. 
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during the baseline period “raised the bar” on the program considerably in terms of what was 
required to produce future employment gains. 5 

The employment rates of the Jobs-Plus group and comparison group remained similar 
until early 2000, when the Jobs-Plus group began to move ahead slightly. This small advantage 
fluctuated over time but was sustained for the rest of the follow-up period. Therefore, the graph 
provides visual evidence of a small program-induced increase in quarterly employment rates for 
all sites combined. 

Panel B of Figure 4.2 displays a more pronounced pattern for the three stronger imple- 
mentation sites. This suggests that Jobs-Plus produced larger-than-average increases in em- 
ployment rates for sites that implemented the program relatively well. 

Table 4.2 presents the results of a statistical analysis of the employment rates in Figure 
4.2. Findings are presented in terms of annual average quarterly employment rates — the mean 
percentage of sample members employed per quarter. For example, the 1998 rate of 50.3 per- 
cent for the Jobs-Plus group from all sites combined (Panel A) implies that half its members 
were employed during a quarter. 

Results for employment rates follow a pattern over time that is similar to that for earn- 
ings. For example, the average quarterly employment rate for all sites combined rose from 50.3 
percent in 1998 to 57.3 in 2000 and then fell somewhat as the national recession set in. How- 
ever, the estimated effects of Jobs-Plus on employment rates are less striking than those for 
earnings. This is especially true for all sites combined: From 2000 to 2003, Jobs-Plus was esti- 
mated to have increased rates by only 1.2 percentage points — or 2.3 percent of what they 
would have been without the program. This finding is not statistically significant, meaning it is 
not outside the range of variation that could have occurred by chance due to random period-to- 
period fluctuation. 

Findings for the three stronger implementation sites (Panel B of Table 4.2) indicate a 
program effect on employment rates that is four times that for all sites combined but is still not 
statistically significant — leaving considerable uncertainty about the result. For these sites, the 
data suggest that Jobs-Plus increased employment rates during the period from 2000 to 2003 by 
4.6 percentage points, from 49.0 percent in the absence of the program to 53.6 percent in its 
presence. This implies that Jobs-Plus increased employment rates by 9.4 percent of what they 
would have been otherwise, which, in turn, implies that the program increased the number of 
persons employed per quarter by 9.4 percent. (Findings presented later in this chapter indicate 



5 It is also important to note that because employment impacts are based on quarterly UI data, the analysis 
of employment rates does not capture any impact that Jobs-Plus may have had in reducing the length of time 
out of work within a given quarter. 
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The Jobs-Plus Demonstration 

Table 4.2 



Pooled Average Annual Outcomes for Quarterly Employment Rates, 
by Site Combination and Follow-Up Period (1998 Cohort) 



Follow-Up Period 


Observed 
Outcome with 
Jobs-Plus 


Estimated 
Effect of 
Jobs-Plus 


Estimated 
Outcome Without 
Jobs-Plus 


Estimated 
Percentage 
Change 
in Outcome Due to 
Jobs-Plus 


A. All Sites Combined (%) 










1998 


50.3 


0.5 


49.8 


1.0 


1999 


55.7 


1.0 


54.7 


1.8 


2000 


57.3 


2.2 


55.1 


4.0 


2001 


54.8 


2.1 


52.7 


4.0 


2002 


50.8 


0.4 


50.4 


0.8 


2003 


48.2 


0.1 


48.1 


0.2 


2000-2003 


52.8 


1.2 


51.6 


2.3 


B. Stronger Implementation Sites Combined (%) 






1998 


49.0 


2.9 


46.1 


6.3 


1999 


54.9 


2.8 


52.1 


5.4 


2000 


57.0 


4.7 


52.3 


9.0 


2001 


56.0 


5.8 


50.2 


11.6 


2002 


51.5 


3.3 


48.2 


6.8 


2003 


50.1 


4.5 


45.6 


9.9 


2000-2003 


53.6 


4.6 


49.0 


9.4 



SOURCES: MDRC calculations using data from housing authority tenant (50058) records and state 
Unemployment Insurance (UI) wage records. 



NOTES: The 1998 cohort includes all residents of a Jobs-Plus development in October 1998 who were 
between 21 and 61 years old and were not listed as disabled by their public housing authority. 

Statistical significance levels are indicated as *** = 1 percent; ** = 5 percent; and * = 10 percent. 

There were 2,123 persons in the Jobs-Plus group and 2,651 persons in the comparison group for all sites 
combined. There were 1,088 persons in the Jobs-Plus group and 1,209 persons in the comparison group for 
the stronger implementation sites combined. 
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that Jobs-Plus increased the quarterly employment rates of key demographic subgroups from 
the stronger implementation sites by amounts that are substantial and statistically significant. 
Hence, it is reasonable to infer that the overall estimate for these sites combined represents a 
real program impact, even though it is not statistically significant.) 

A further question to address when assessing these results is “How much of the pro- 
gram effect on earnings from 2000 to 2003 in the stronger implementation sites was due to in- 
creased employment rates?” 6 As noted, the data suggest that Jobs-Plus increased average earn- 
ings per member of the 1998 cohort by 14.1 percent and increased the number of cohort mem- 
bers employed by 9.4 percent. These findings imply that two-thirds (9.4 percent / 14.1 percent) 
of the earnings gain produced by Jobs-Plus was due to an increase in the number of persons 
employed. 7 The remaining third of the increase in earnings produced by Jobs-Plus represents a 
mix of increases in the number of weeks employed per period (employment stability), the num- 
ber of hours worked per week employed (employment intensity), and wages paid per hour 
worked (the skill level of employment). Given available data, there is no way to identify the 
separate contributions of these factors. 8 

Large Variations in Earnings Impacts and Employment Impacts Across 
Sites 

In order to understand the preceding findings more fully, it is important to consider how 
they vary across sites in the study. For this purpose, Figure 4.3 graphs average quarterly earn- 
ings for the Jobs-Plus group and for the comparison group from each site separately. 9 These 
graphs clearly illustrate that baseline earnings at all sites rose rapidly for the Jobs-Plus group 
and the comparison group and that their baseline earnings trajectories were well matched. The 
main difference between baseline findings for individual sites and those for sites combined is 
that individual sites have greater random fluctuation (often called “noise”) due to their smaller 
samples. After Jobs-Plus was launched, the relationship between earnings for the Jobs-Plus 
group and earnings for the comparison group varied substantially across sites. 

In Baltimore (Panel A), the earnings trajectories of the Jobs-Plus group and the com- 
parison group remained almost identical during the six-year follow-up period after Jobs-Plus 

^Because the three stronger implementation sites had the clearest earnings impacts, they provide the best 
basis for explaining how earnings impacts were produced by Jobs-Plus. 

Expressing the earnings and employment rate changes of 14.1 percent and 9.4 percent as ratios equal to 
1.141 and 1.094, it follows that 1.141 equals (1.094)a, where x equals 1.043. This implies a 4.3 percent change 
in earnings per person employed. 

Although no information exists on how Jobs-Plus affected the characteristics of jobs held by residents. 
Appendix Table E.l presents data from the follow-up surveys in three sites — Baltimore, Dayton, and St. Paul 
— that describe the jobs held. 

9 Similar graphs for quarterly employment rates by site are presented in Appendix Figure E. 1 . 
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Mean Quarterly Earnings Mean Quarterly Earnings 

(in 2003 dollars) (in 2003 dollars) 



Figure 4.3 (continued) 




(continued) 





Mean Quarterly Earnings Mean Quarterly Earnings 

(in 2003 dollars) (in 2003 dollars) 



Figure 4.3 (continued) 
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◄ Baseline Follow-Up — 
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* Baseline Follow-Up 



(continued) 



Figure 4.3 (continued) 

SOURCES: MDRC calculations using data from housing authority tenant (50058) records and state 
Unemployment Insurance (UI) wage records. 

NOTES: The 1998 cohort includes all residents of a Jobs-Plus development or a comparison 
development in October 1998 who were between 21 and 61 years old and were not listed as disabled 
by their public housing authority. 

‘Unemployment Insurance (UI) wage data for the second quarter of 1993 were not available for 
Dayton. 



was launched — strong evidence that the program in Baltimore did not have an earnings effect. 
This result is consistent with the exceptionally low penetration rate of the site’ s Jobs-Plus pro- 
gram, where, for example, only 12 percent of the 1998 cohort members used the rent incentives 
(versus 28 percent to 67 percent in the other sites). (As Chapter 3 shows, use of the rent incen- 
tives was highest in the sites that most strongly implemented Jobs-Plus overall. Thus, the incen- 
tives take-up rate is a convenient summary indicator of the quality of program implementation.) 

In Chattanooga (Panel B), the earnings trajectory of the Jobs-Plus group was slightly 
below that of the comparison group for much of the baseline period and then somewhat below 
again during the latter part of the follow-up period. The pattern yields no indication that Jobs- 
Plus increased the earnings of residents from this site, which is consistent with the documented 
weak implementation of Jobs-Plus in Chattanooga and the fact that it never became a fully func- 
tioning program there. 

In Dayton (Panel C), the earnings trajectories of the Jobs-Plus group and the compari- 
son group continued to be very similar during the first two years after Jobs-Plus was launched. 
Soon thereafter, however, earnings for the Jobs-Plus group began to exceed those for the com- 
parison group. The earnings advantage for the Jobs-Plus group was sustained throughout the 
remainder of the follow-up period, although its magnitude fluctuated. These positive findings 
are especially impressive in light of the high rates of resident mobility in Dayton, where 48 per- 
cent of the 1998 cohort members in the Jobs-Plus group moved away within two years (versus 
17 percent to 41 percent in the other sites). 

In Los Angeles (Panel D), the earnings trajectory of the Jobs-Plus group shifted sharply 
above that of the comparison group two years after Jobs-Plus was launched. 10 This pronounced 



10 The findings presented in this chapter for Los Angeles represent the Jobs-Plus program at William Mead 
Homes and its comparison development. Appendix Figure E.2 illustrates the earnings and employment histo- 

(continued) 
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and prolonged earnings gain matches the timing of an aggressive revival of the Los Angeles 
Jobs-Plus program, following its earlier near-collapse. The finding is also consistent with the 
especially high penetration rate of the site’s program, with 61 percent of the 1998 cohort mem- 
bers using the rent incentives. Furthermore, it is consistent with the site’s especially low rate of 
resident mobility, with only 17 percent of the 1998 cohort members in the Jobs-Plus group 
moving away within two years. 

In St. Paul (Panel E), which was the only site to implement rent incentives in the first 
year after Jobs-Plus was launched (1998), the earnings trajectory of the Jobs-Plus group moved 
immediately above that of the comparison group. The gap widened substantially in later years, 
as program group members pulled further ahead of comparison group members. This pattern 
reflects the largest estimated program effects on earnings for any site. And this finding is consis- 
tent with the fact that St. Paul had the highest program penetration rate, with 67 percent of the 
1998 cohort members in its Jobs-Plus group participating in rent incentives. In addition, it is 
consistent with the site’s relatively low rates of resident mobility, with only 27 percent of the 
1998 cohort members in the Jobs-Plus group moving out in two years. 

Lastly, in Seattle (Panel F), the earnings trajectory of the Jobs-Plus group stayed close 
to that of the comparison group until 2000, when it began to pull ahead, and 2001, when it was 
well above that of the comparison group. This earnings advantage disappeared by the next year, 
however, most likely because of the disruption and dislocation produced by the HOPE VI reno- 
vation and relocation program that was getting under way. It appears that as Jobs-Plus fully ma- 
terialized in the site, it began to generate substantial earnings gains for residents. But these gains 
were overtaken by events as residents were relocated in large numbers through HOPE VI. The 
expansion of employment services and the introduction of new rent incentives in the compari- 
son site as part of the housing authority’s Moving to Work demonstration may have also con- 
tributed to the decline in Seattle’s impacts. 11 

To summarize these site-specific stories and make their findings more concrete and pre- 
cise, Figure 4.4 presents statistical estimates of the site-by-site effects of Jobs-Plus on earnings 
for each follow-up year. For Jobs-Plus in Baltimore, there is no sign of a program effect on 
earnings during any year. All estimated effects are near zero, and none are statistically signifi- 
cant. Hence, the program in this site did not achieve its work-related goals. For Jobs-Plus in 
Chattanooga, there also was no sign of a positive effect on earnings. Annual impact estimates 
varied from slightly positive to slightly negative after the program was launched and all but one 
were not statistically significant. 



ries for residents of the other Jobs-Plus development in Los Angeles, Imperial Courts, which did not have a 
comparison development. 

1 'See Liebow et al., 2004. 
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Figure 4.4 (continued) 



SOURCES: MDRC calculations using data from housing authority tenant (50058) records and state 
Unemployment Insurance (UI) wage records. 

NOTES: All earnings effects are reported in 2003 dollars. 

The 1998 cohort includes all residents of a Jobs-Plus development or a comparison 
development in October 1998 who were between 21 and 61 years old and were not listed as disabled 
by their public housing authority. 

Statistical significance levels are indicated as *** =1 percent; ** = 5 percent; * =10 percent. 

Sample sizes are 345 in the program group and 610 in the comparison group in Baltimore; 278 
in the program group and 502 in the comparison group in Chattanooga; 340 in the program group and 
595 in the comparison group in Dayton; 446 in the program group and 385 in the comparison group 
in Los Angeles; 302 in the program group and 229 in the comparison group in St. Paul; and 412 in 
the program group and 330 in the comparison group in Seattle. 



For Jobs-Plus in Dayton, Los Angeles, and St. Paul (the stronger implementation sites), 
there are substantial estimated program effects on earnings during every year from 2000 to 
2003. 12 Program-induced earnings gains ranged from $734 to $1,239 (10.0 percent to 19.7 per- 
cent) in Dayton; from $548 to $1,415 (7.1 percent to 19.0 percent) in Los Angeles; and from 
$937 to $2,128 (10.2 percent to 22.1 percent) in St. Paul. Therefore, there is considerable evi- 
dence that Jobs-Plus was successful in achieving its work-related goals in these sites. 

Lastly, for Jobs-Plus in Seattle, there were signs of substantial early program effects 
($615 and $1,050 in 2000 and 2001). Thus, when a relatively strong program was in effect, the 
site was able to achieve its work-related goals. But this initial success was not sustained when 
the Jobs-Plus development underwent a HOPE VI transformation. As this transformation got 
under way, the extra earnings that had been stimulated by Jobs-Plus disappeared, and earnings 
levels returned to what they would have been without the program. 

To complete this part of the analysis, Figure 4.5 presents site-by-site estimates of the 
impacts of Jobs-Plus on mean quarterly employment rates by follow-up year. The estimates for 
Jobs-Plus in Baltimore and Chattanooga fluctuate just above and below zero, with none that are 
large and only one that is statistically significant. The estimates for Dayton, Los Angeles, and 



p During 2000, as the Los Angeles Jobs-Plus program was being rebuilt, its estimated earnings effect was 
substantial in magnitude but not yet statistically significant. 
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The Jobs-Plus Demonstration 



Figure 4.5 

Average Annual Effects of Jobs-Plus on Employment Rates, 
by Site (1998 Cohort) 

1998 1999 2000 2001 2002 2003 



Baltimore 



- 0 . 1 % 



- 4 . 3 % 



1 . 7 % 




- 3 . 8 % 



Chattanooga 



0 . 8 % 



2 - 6 % 1 . 6 % 



- 3 . 5 % 



- 4 . 0 % * 



Dayton 



0 . 9 % 



3 . 8 % 




5 . 1 % ** 




1 . 1 % 



1 . 2 % 



2 . 8 % 



2 . 7 % 



2 . 2 % 



Los Angeles 



- 1 . 1 % 



St. Paul 



1.6% 



5 . 3 % 



6 . 7 % 



1 . 2 % 





3 . 0 % 



4 . 7 % 




Seattle 



2 . 1 % 



0 . 5 % 



0.2% 



2 . 0 % 



0 . 2 % 



- 2 . 3 % 



(continued) 
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Figure 4.5 (continued) 

SOURCES: MDRC calculations using data from housing authority tenant (50058) records and state 
Unemployment Insurance (UI) wage records. 

NOTES: The 1998 cohort includes all residents of a Jobs-Plus development or a comparison 
development in October 1998 who were between 21 and 61 years old and were not listed as disabled 
by their public housing authority. 

Statistical significance levels are indicated as *** =1 percent; ** = 5 percent; * =10 percent. 
Sample sizes are 345 in the program group and 610 in the comparison group in Baltimore; 278 in 
the program group and 502 in the comparison group in Chattanooga; 340 in the program group and 
595 in the comparison group in Dayton; 446 in the program group and 385 in the comparison group in 
Los Angeles; 302 in the program group and 229 in the comparison group in St. Paul; and 412 in the 
program group and 330 in the comparison group in Seattle. 



St. Paul suggest a pattern of positive effects, with the largest effects in Dayton and St. Paul and 
the only statistically significant effects in Dayton. The findings for Seattle are near zero 
throughout the follow-up period. On balance, the three sites with the largest and most sustained 
program effects on earnings exhibit some evidence of a corresponding but proportionately 
smaller effect on employment rates. For both the sites with no earnings effects (Baltimore and 
Chattanooga) and the site with earnings effects that are not sustained (Seattle), there is little or 
no sign of a program effect on employment rates. 

Large Impacts for Early and Later Cohorts of Residents 

Results presented so far in this chapter are for persons who were residents of a Jobs- 
Plus development when the program was launched. This group was used as the primary basis 
for assessing the effects of Jobs-Plus on individuals because its composition was determined 
before the program existed and thus could not have been influenced by it. 13 Thus, it provides a 
valid comparison of the Jobs-Plus group and the comparison group. Nevertheless, many mem- 
bers of the 1998 cohort moved away before implementation of Jobs-Plus was completed — be- 
fore they could be exposed to the full program. Therefore, it is possible that findings for the 
1998 cohort understate the effects of Jobs-Plus. To address this possibility, the analysis was re- 
peated for a later cohort of working-age, nondisabled adults who were residents of a Jobs-Plus 
development or a comparison development in October 2000 — the 2000 cohort. 14 Members of 



13 In statistical parlance, the composition of the 1998 cohort is “exogenous” to Jobs-Plus. 

14 Because it is possible that Jobs-Plus could have influenced the composition of the 2000 cohort at the 
Jobs-Plus developments by inducing selective in-migration or out-migration, findings for this cohort are used 
only for comparative purposes. However, there is no empirical evidence that Jobs-Plus actually influenced its 
composition. 
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the 1998 cohort who did not move away were also members of the 2000 cohort, meaning that 
the two cohorts overlap somewhat. Nevertheless, they differ considerably due to the many 
members of the 1998 cohort who moved away before October 2000 and the many members of 
the 2000 cohort who moved in after October 1998. Specifically, 59.9 percent of the 1998 cohort 
members and 62.1 percent of the 2000 cohort members were in both cohorts. 15 

Table 4.3 compares estimates of the impacts of Jobs-Plus on residents’ earnings for the 
two cohorts from all sites except Seattle. 16 These findings represent the average effects of Jobs- 
Plus on annual earnings from 2000 to 2003, after the program had been implemented. Findings for 
the two cohorts tell the same story. For example, estimated program effects in Baltimore and 
Chattanooga are near zero for both cohorts. Thus, underexposure to the program because of resi- 
dent mobility when it was being implemented cannot explain its lack of success at these sites. In 
Dayton, Los Angeles, and St. Paul, estimated program effects are positive, large, and statistically 
significant for both cohorts. In one of these sites, the estimate is larger for the 2000 cohort; in an- 
other, the estimates are virtually the same; and in another, the estimate is smaller for the 2000 co- 
hort. Therefore, it does not appear that using results for the 1998 cohort understates the average 
effects of Jobs-Plus because of “dosage dilution” caused by resident mobility. 17 

Large Impacts for All Subgroups and Especially Large Impacts for Some 

The preceding findings indicate that Jobs-Plus increased earnings substantially at the 
three sites where it was implemented strongly. Table 4.4 illustrates how these impacts differed 
for subgroups of the 1998 cohort that could be identified using available data. 18 The first column 
of the table lists the estimated effect of Jobs-Plus on average annual earnings from 2000 to 
2003, after program implementation was completed. These findings are presented in 2003 dol- 
lars. The second column presents each estimated program effect as a percentage of what earn- 
ings would have been without Jobs-Plus. The third column indicates whether the estimates of 
program effects for subgroups defined in terms of a particular characteristic (for example, 



15 These figures are for all sites except Seattle. 

16 A program effect is not reported for the 2000 cohort in Seattle because evidence suggests that HOPE VI 
influenced its composition substantially. 

17 ln addition, repeating the analysis for the 2000 cohort provides an internal replication of the analysis for 
the 1998 cohort, which helps to establish the robustness and generalizability of the findings. 

ls Results for each subgroup were obtained by (1) computing its mean earnings for the Jobs-Plus group and 
for the comparison group by site and quarter; (2) computing the three-site mean-of-mean earnings for the Jobs- 
Plus subgroup and the comparison subgroup by quarter, weighting each site equally; and (3) conducting a 
comparative interrupted time-series analysis of the pooled quarterly earnings results for the Jobs-Plus subgroup 
and comparison subgroup at the three sites. Weighting the subgroup samples from all sites equally eliminates 
potential confounds between site-specific program effects and subgroup sample sizes. In addition, because 
estimates of program effects are obtained for each subgroup separately, their average differs from that for the 
full sample due to random estimation error. 
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The Jobs-Plus Demonstration 
Table 4.3 



Average Annual Effects of Jobs-Plus on Earnings from 2000 to 2003: 
1998 and 2000 Cohorts for Each Site 



Site 


Earnings Effect for 1998 Cohort 


Earnings Effect for 2000 Cohort 


Dollars 


Percentage 


Dollars 


Percentage 


Baltimore 


-189 


-2.7 


168 


2.5 


Chattanooga 


-224 


-3.3 


24 


0.4 


Dayton 


895 *** 


13.0 


1,189 *** 


19.4 


Los Angeles 


1,120 *** 


14.8 


1,076 *** 


14.8 


St. Paul 


1,492 *** 


15.4 


*** 


8.0 


Seattle 


394 ** 


4.0 


NA 


NA 



SOURCES: MDRC calculations using data from housing authority tenant (50058) records and state 
Unemployment Insurance (UI) wage records. 

NOTES: All earnings effects are reported in 2003 dollars. 

Statistical significance levels are indicated as *** = 1 percent; ** = 5 percent; and * = 10 
percent. 

The number of persons in the 1998 and 2000 cohorts, respectively, were 955 and 888 for 
Baltimore; 780 and 825 for Chattanooga; 935 and 684 for Dayton; 831 and 904 for Los Angeles; 
and 531 and 523 for St. Paul. There were 742 persons in the 1998 cohort in Seattle. 



TANF receipt or not) are statistically significantly different from each other. These findings tell 
a story that demonstrates the robustness of the Jobs-Plus program effects and provides a deeper 
understanding of them. 

Table 4.4 indicates that once Jobs-Plus was implemented, it produced large earnings 
gains for every subgroup and very large earnings gains for many of them. 19 This demonstrates 
that the program produced meaningful earnings gains for many different types of public housing 
residents, not just a small, idiosyncratic subgroup. In this regard the findings are quite robust. 



l9 Subgroup estimates of program effects during 1998-1999 (not presented here) indicate that Jobs-Plus did 
not produce large program effects for any subgroups while the program was being implemented. For most sub- 
groups, this early effect was very small, and for only one was it statistically significant. 
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The Jobs-Plus Demonstration 
Table 4.4 



Average Annual Effects of Jobs-Plus on Earnings for 
the Stronger Implementation Sites Combined, by Subgroup (1998 Cohort) 



Subgroup 


Program Effect 
2000-2003 ($) 


Percentage 
Change (%) 


Statistical 
Significance of 
the Differences 


Sample 

Sizes 


Did not receive TANF in 1998 


1,654 *** 


17.9 


tft 


1,157 


Received TANF in 1998 


761 ** 


10.7 




1,140 


Had lived in development less than 4 years 


650 *** 


8.4 


tft 


1,382 


Had lived in development at least 4 years 


1,818 *** 


21.1 




915 


Age 










21-24 years 


1 921 *** 


22.1 


tft 


414 


25 - 34 years 


1,323 *** 


14.2 




807 


35 - 61 years 


756 *** 


10.7 




1,076 


Had been employed less than 3 of past 8 quarters 


1 427 *** 


37.8 




1,050 


Had been employed at least 3 of past 8 quarters 


882 * 


7.2 




1,247 


Moved out by October 2000 


1,505 *** 


15.9 




799 


Did not move out by October 2000 


1,082 *** 


14.4 




1,498 



SOURCES: MDRC calculations using data from housing authority tenant (50058) records, state Unemployment 
Insurance (UI) wage records, and state AFDC/TANF records. 

NOTES: Findings are reported in 2003 dollars, and those for each site in the analysis — Dayton, Los Angeles, and 
St. Paul — are weighted equally. 

Statistical significance levels are indicated as *** = 1 percent; ** = 5 percent; and * = 10 percent. 

Statistical significance of the differences in levels are indicated as ftt = 1 percent; tf = 5 percent; and t = 

10 percent. 

Members of the welfare recipient subgroup received a welfare payment in 1998; members of the non- 
recipient subgroup did not. 



However, there is also enormous variation in the earnings impacts of Jobs-Plus. These 
gains range from a low of $650 per year for cohort members who had lived in their develop- 
ment for less than four years before Jobs-Plus began (recent arrivals) to a high of $1,921 per 
year for those who were 21 to 24 years old at the time (young adults). In percentages, the gains 
ranged from a low of 7.2 percent for cohort members who were employed for at least three of 
the past eight quarters before Jobs-Plus began (less disadvantaged persons) to a high of 37.8 
percent for those who were employed for less than three quarters (more disadvantaged persons). 
Consider the results for each set of subgroups. 
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Large Differences for Subgroups Defined in Terms of Welfare Receipt 

Perhaps the most important subgroup differences in Table 4.4 are those for recent TANF 
recipients versus nonrecipients. These findings are important in their own right and provide in- 
sights into how the effects of Jobs-Plus should be interpreted. They indicate that, on average, dur- 
ing the four years after Jobs-Plus was implemented in Dayton, Los Angeles, and St. Paul, the pro- 
gram increased the earnings of cohort members who were not on welfare (were not receiving 
TANF) when Jobs-Plus was launched — by $1,654 per year, or 17.9 percent. For cohort members 
who were on welfare when the program was launched, Jobs-Plus increased average annual earn- 
ings by $761 per year, or 10.7 percent. 20 Both of these estimates are statistically significantly dif- 
ferent from zero. In addition, they are statistically significantly different from each other. 

The most likely explanation for this subgroup difference in program impacts is that the 
Jobs-Plus/comparison group “service differential” was reduced for welfare recipients by employ- 
ment-related services and incentives that were available to welfare recipients in the comparison 
group from welfare reform initiatives that were under way when Jobs-Plus was being imple- 
mented. There were no existing counterparts to these welfare-to-work programs for cohort mem- 
bers who were not welfare recipients. It is therefore likely that the service differential produced by 
Jobs-Plus for welfare recipients in the 1998 cohort was smaller than that for nonrecipients. Unfor- 
tunately, there is no way to directly observe this subgroup difference because it was not feasible to 
collect data on employment-related services received by a representative sample of the 1998 co- 
hort during the follow-up period for Jobs-Plus, given that so many moved away. (Recall that the 
follow-up survey of residents was limited to those living in the development in 2003.) However, 
the time-series earnings graphs in Figure 4.6 provide some indirect clues. 

The graph in Panel A of Figure 4.6, for nonrecipients of TANF, indicates that, from 
1994 through 1999, the earnings trajectories of the Jobs-Plus group and the comparison group 
were rapidly rising and very similar to each other. After 2000, with the onset of the national re- 
cession, earnings for the comparison group fell, whereas earnings for the Jobs-Plus group con- 
tinued to rise rapidly. Thus, Jobs-Plus produced a very large earnings gain for the subgroup not 
receiving welfare. 



20 The ratio of earnings effects in dollars for the two welfare subgroups was 1,654/761, or 2.17, whereas 
the ratio in percentages was 17.9/10.7, or 1.67. The difference in these ratios reflects the fact that earnings in 
the absence of Jobs-Plus were much higher for nonrecipients of welfare than for recipients. 
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The Jobs-Plus Demonstration 



Figure 4.6 

Average Quarterly Earnings for Members of the 1998 Cohort Who Were and 
Were Not Receiving Welfare (Stronger Implementation Sites) 





0 / 0 / 0 / 0 / 0 / 0 / 0 / 0 / 0 / 0 / 0 / 0 / 
◄ Baseline Follow-Up ► 





0 / 0 / 0 / 0 / 0 / 0 / 0 ' 0 / 0 ' 0 ' 



a a 



Baseline 



Follow-Up 



SOURCES: MDRC calculations using data from housing authority tenant (50058) records, state 
Unemployment Insurance (UI) wage records, and state AFDC/TANF records. 

NOTE: The 1998 cohort includes all residents of a Jobs-Plus development or a comparison 
development in October 1998 who were between 21 and 61 years old and were not listed as disabled 
by their public housing authority. 
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The graph in Panel B of Figure 4.6 is for TANF recipients in the Jobs-Plus group and 
the comparison group. Earnings for these subgroups remained almost constant at a very low 
level until 1997, when they began to rise rapidly. 21 This rise stopped after 2000 for the compari- 
son group but continued somewhat for the Jobs-Plus group. Thus, Jobs-Plus produced a moder- 
ately large earnings gain for the subgroup receiving welfare. 

So why did earnings for welfare recipients not increase until 1997 — about when wel- 
fare reforms began — and then increase sharply for both the Jobs-Plus group and the compari- 
son group? One explanation is that employment services and work incentives provided by wel- 
fare reforms produced much of the earnings increases for both groups, thereby reducing the 
margin for Jobs-Plus to make a difference. This would not have been the case for residents who 
were not welfare recipients. 

Large Differences for Subgroups Defined in Terms of Prior Duration of 
Residence 

The findings presented in Table 4.4 indicate that the earnings gains produced by Jobs- 
Plus were much larger for cohort members who had lived in their development for a number of 
years than for recent arrivals. Specifically, it appears that Jobs-Plus increased the earnings of 
persons who had lived in their development for at least four years by $1,818 per year, or 21.1 
percent. This earnings gain was $650 per year, or 8.4 percent, for persons who had moved into 
their development more recently. The difference between these estimates for the two subgroups 
was quite large and is highly statistically significant. One possible explanation for the difference 
is that residential stability (even in public housing) facilitates other positive changes. Thus, a 
stable housing situation might make it easier for some persons to participate in an employment 
initiative and take the steps needed to find and keep new employment. These conditions might 
be less prevalent for members of households that had moved recently and experienced the dis- 
ruption that comes with doing so. 

An alternative explanation is that longer-term public housing residents had lower past 
earnings than did recent arrivals and thus a greater margin for improvement. However, the data 
indicate that the reverse is true. Therefore, longer-term residents experienced larger program 
effects even though they had a smaller margin for improvement. 



“'One explanation for the rapid rise in earnings after 1998 is that persons who received welfare that year 
overrepresent those who had just experienced an unusually bad time and would rebound naturally. In statistical 
parlance, this implies that the rapid rise is a “regression artifact.” But if this were the case, average earnings 
would also rise when going backward in time from 1998 — which did not occur. 
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Large Differences for Subgroups Defined in Terms of Age 

Table 4.4 also indicates that the earnings gains produced by Jobs-Plus declined substan- 
tially and systematically as residents’ age increased. These gains were $1,921, or 22.1 percent, 
for cohort members who were 21 to 24 years old when Jobs-Plus was launched; $1,323, or 14.2 
percent, for those who were 25 to 34 years old; and $756, or 10.7 percent, for those who were 
35 to 61 years old. One possible explanation for these large statistically significant differences is 
that younger residents are more flexible and can better adapt to changes made possible by an 
employment initiative such as Jobs-Plus. 

Smaller Differences for Subgroups Defined in Terms of Past Employment and 
Future Mobility 

Estimates of the effects of Jobs-Plus for subgroups defined in terms of recent past em- 
ployment and future residential mobility differ by much less than do those for the preceding 
categories of subgroups. In addition, these differences are not statistically significant and thus 
might reflect random estimation error. Findings for the employment subgroups provide little 
evidence that Jobs-Plus was appreciably more or less effective for residents with a strong or 
weak recent attachment to the labor market. 22 Thus, they do not suggest a more refined targeting 
strategy for the program. Findings for the mobility subgroups suggest that program effects for 
residents who stayed in their development (and thus may have contributed to its economic im- 
provement) were roughly comparable to those for residents who moved away (and took their 
economic benefits with them). They suggest that the program’s positive effects for individual 
residents might translate into positive effects for their developments (discussed in Chapter 5). 23 

Fastly, to more fully interpret the preceding subgroup findings, it is useful to consider 
the degree to which subgroup characteristics are associated with each other. If this association is 
high, then subgroups defined in terms of one characteristic (for example, welfare receipt) will 
largely overlap subgroups defined in terms of other characteristics (for example, duration of 
tenancy). And, if so, differences in program effects might be related to one characteristic be- 
cause they are related to another. 

Perhaps most important in this regard is the degree to which welfare receipt is inde- 
pendent of the other subgroup characteristics. A simple cross-tabulation of welfare receipt ver- 



22 The especially large percentage increase in earnings (37.8 percent) for the less employed subgroup re- 
flects these residents’ especially low earnings levels in the absence of Jobs-Plus. 

23 Unlike all the other subgroups in Table 4.4, those defined in terms of future mobility are based on behav- 
ior that occurred after Jobs-Plus was launched and, thus, in theory, is behavior that could have been affected by 
the program. Estimates (not presented in this report) suggest, however, that resident mobility was not affected 
by the program. 
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sus each subgroup characteristic indicates that it is largely independent of the others. 24 Hence, 
subgroups defined in terms of welfare receipt do not substantially overlap those defined in terms 
of other characteristics. This is further evidence that the difference in program effects for wel- 
fare subgroups may reflect differences in services available to comparison group members 
rather than differences in their characteristics. Nevertheless, it is possible that welfare recipients 
in the study differ from nonrecipients in unobserved ways that caused the two groups to respond 
differently to Jobs-Plus. 

Dramatic Findings for Subgroups Within Stronger Implementation Sites 

So far, the analysis of subgroup impacts has focused on the three stronger implementa- 
tion sites combined. This was done to ensure adequate sample sizes for each subgroup. It was 
also done to limit the number of impact findings presented in order to reduce the potential for 
“information overload.” However, further important insights into the effectiveness of Jobs-Plus 
become apparent when estimates of its impacts are presented separately for the primary demo- 
graphic subgroups from each of the three stronger implementation sites, which have adequate 
sample sizes to support separate analyses. These subgroups are African-American women in 
Dayton, immigrant Latin American men and Latin American women in Los Angeles, and im- 
migrant Southeast Asian men and Southeast Asian women in St. Paul. 25 

Table 4.5 presents findings for these subgroups. The first two columns in the table report 
estimates of the impacts of Jobs-Plus on average annual earnings from 2000 to 2003 in dollars and 
as a percentage of what earnings would have been without the program. The two right-hand col- 
umns report estimates of the impacts of Jobs-Plus on average quarterly employment rates from 
2000 to 2003 in percentage points and as a percentage of what employment rates would have been 
without the program. (Note that the rightmost column represents percentages of percentages.) 



24 The phi coefficient (which summarizes the association between two categorical variables) for residents’ 
welfare receipt versus their duration of prior tenancy, age, prior employment, and future mobility was, respec- 
tively, -0.087, 0.140, 0.096, and 0.061. 

25 In Dayton, 75.8 percent of the 1998 cohort members were identified as black, non-Hispanic women ac- 
cording to their housing authority records. Moreover, 99.8 percent of the black, non-Hispanic women in the 
Dayton baseline survey sample said that they were bom in the United States. In Los Angeles, 78.6 percent of 
the 1998 cohort members were identified as Hispanic according to their housing authority records, and 90.7 of 
the Hispanic members of the Los Angeles baseline survey sample said that they were bom outside the United 
States (approximately two-thirds were bom in Mexico, and one-third were bom in El Salvador, Guatemala, 
Honduras, and other Central American countries). In St. Paul, 52.8 percent of the 1998 cohoit members were 
identified as Asians or Pacific Islanders, according to their housing authority records, and 93.7 percent of the 
Asians and Pacific Islanders in the St. Paul baseline survey sample (almost all of whom were Hmong) said that 
they were born in Laos or Cambodia. 



94 




The Jobs-Plus Demonstration 
Table 4.5 

Average Annual Effects of Jobs-Plus on Earnings and Quarterly Employment 
Rates from 2000 to 2003 for the Largest Demographic Subgroups in the Stronger 

Implementation Sites (1998 Cohort) 

Effect on Earnings Effect on Employment Rate 

Percentage Percentage Percentage 

Site and Subgroup Dollars Change Points Change 



Davton 

Black, non-Hispanic women 
(n = 286 and 423) 


\ 144 *** 


16.3 


4 7 *** 


8.1 


Los Angeles 


Hispanic men (n = 111 and 102) 


3,248 *** 


28.3 


10.8 *** 


18.6 


Hispanic women (n = 235 and 205) 


649 


10.4 


1.4 


3.6 


St. Paul 


Southeast Asian men (n = 72 and 59) 


2,129 * 


20.6 


2.2 


4.7 


Southeast Asian women (n = 98 and 78) 


1 798 *** 


23.4 


12.8 *** 


38.6 



SOURCES: MDRC calculations using data from housing authority tenant (50058) records and state 
Unemployment Insurance (UI) wage records. 



NOTES: The numbers of persons in the Jobs-Plus sample and comparison sample (respectively) are listed in 
parentheses after the name of each subgroup. 

Statistical significance levels are indicated as *** = 1 percent; ** = 5 percent; and * = 10 percent. 

The 1998 cohort includes all residents of a Jobs-Plus development or a comparison development in October 
1998 who were between 21 and 61 years old and were not listed as disabled by their public housing authority. 



The overall pattern of impact estimates in the table illustrates three main points about 
the effectiveness of Jobs-Plus when it is implemented relatively well. First, note that large earn- 
ings gains were generated for four of the five subgroups . 26 This indicates that Jobs-Plus was ca- 
pable of generating large earnings gains for public housing residents who vary enormously in 
their backgrounds and personal experiences, ranging from native-born African-American 
women to immigrant groups from two different parts of the world. Such findings reinforce the 
conclusion that Jobs-Plus can be effective for many different types of public housing residents. 



26 The second-largest demographic subgroup in the St. Paul sample was made up of 104 black, non- 
Hispanic women, half of whom lived in the Jobs-Plus development and half of whom lived in the comparison 
development. The erratic nature of the time-series data for this small sample (see Appendix Figure E.3) made it 
infeasible to estimate the impacts of Jobs-Plus for this subgroup. 
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Second, note that the findings for quarterly employment rates indicate that Jobs-Plus pro- 
duced increases that were often sizable and statistically significant. This was the case for three of 
the four subgroups that experienced large earnings impacts. Hence, there is strong evidence that 
Jobs-Plus increased employment rates. This reinforces the conclusion that estimates of employ- 
ment impacts for the combined sample of stronger implementation sites presented earlier (which 
are positive but not statistically significant) probably represent real effects of the program. 

Third, note that the findings for earnings and employment rates in Table 4.5 present a 
complex picture of the relationship between the effects of Jobs-Plus on these two outcomes. 
This relationship varies dramatically across demographic subgroups. It covers the full range of 
possibilities from one extreme (for Southeast Asian men in St. Paul), where program-induced 
changes in quarterly employment rates (4.7 percent) comprise a small fraction of the program- 
induced change in earnings (20.6 percent), to the opposite extreme (for Southeast Asian women 
in St. Paul), where program-induced changes in employment rates (38.6 percent) comprise all 
the change in earnings (23.4 percent). 27 Hence, there is no single simple story about how Jobs- 
Plus created earnings gains for public housing residents. The only common theme is that it did 
so for very different subgroups. 

Now consider the specific subgroup findings, starting with those for women. For Afri- 
can-American women in Dayton and immigrant Southeast Asian women in St. Paul, estimates 
of the impacts of Jobs-Plus on both earnings and employment rates are substantial and statisti- 
cally significant. For Hispanic women in Los Angeles, the estimates are much smaller and not 
statistically significant. Thus, Jobs-Plus was effective for at least two of the three groups. 

Figure 4.7 illustrates the quarterly earnings patterns that produced the estimates of earn- 
ings impacts for women. 28 The story in Dayton is one of a consistently close baseline match for 
the Jobs-Plus group and comparison group, followed by a visible and sustained relative improve- 
ment for the Jobs-Plus group after the program was implemented. The story in St. Paul is one 
where baseline earnings for the Jobs-Plus group were consistently below those for the comparison 
group, followed by a visible and sustained reversal of positions after Jobs-Plus was implemented. 
In Los Angeles, the relative positions of the Jobs-Plus group and comparison group fluctuated 
during the baseline period and did not move outside the range of fluctuation after the program was 



27 Taken literally, the estimated program effects on earnings and employment rates for Southeast Asian 
women suggest that all earnings gains for them were due to an increase in the number of persons employed and 
that, furthermore, some of the additional persons employed earned less than the average for those who would 
have been employed in the absence of the program. However, given the substantial random error in all the im- 
pact estimates presented and the fact that earnings impacts were estimated separately from employment im- 
pacts, it is likely that much of the margin by which the 38.6 percentage change for employment rates exceeds 
the 23.4 percentage change for earnings is due to estimation error. 

Appendix Figure E.4 presents corresponding graphs for quarterly employment rates. 



96 




Mean Quarterly Earnings Mean Quarterly Earnings 

(in 2003 dollars) (in 2003 dollars) 



The Jobs-Plus Demonstration 



Figure 4.7 

Average Quarterly Earnings for the Jobs-Plus Group and Its Comparison 
Group, for Women in the Largest Demographic Subgroups in the Stronger 
Implementation Sites (1998 Cohort) 





(continued) 
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Figure 4.7 (continued) 
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SOURCES: MDRC calculations using data from housing authority tenant (50058) records and state 
Unemployment Insurance (UI) wage records. 

NOTE: The 1998 cohort includes all residents of a Jobs-Plus development or a comparison 
development in October 1998 who were between 21 and 61 years old and were not listed as disabled 
by their public housing authority. 

a Unemployment Insurance (UI) wage data for the second quarter of 1993 were not available for 
Dayton. 



implemented. These patterns produced statistically significant estimates of the impacts of Jobs- 
Plus on average annual earnings from 2000 to 2003 equal to $1,144 (or 16.3 percent) for African- 
American women in Dayton and $1,798 (or 23.4 percent) for Southeast Asian women in St. Paul. 
They produced a nonstatistically significant estimate of $649 (or 10.4 percent) for Hispanic 
women in Los Angeles. These findings indicate that Jobs-Plus increased average earnings per per- 
son over four years by a total of $4,576 for African-American women in Dayton, $7,192 for 
Southeast Asian women in St. Paul, and $2,596 for Hispanic women in Los Angeles. 
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Figure 4.8 illustrates the quarterly earnings histories of the two male subgroups, which 
provide dramatic evidence that Jobs-Plus was highly effective for them. 29 The graph in Panel A, 
for Hispanic men from Los Angeles, shows that the baseline earnings trend of the Jobs-Plus 
group was somewhat above that of the comparison group and that both trends fluctuated sub- 
stantially. 30 But, after Jobs-Plus was implemented, earnings for the Jobs-Plus group continued to 
climb, while those for the comparison group leveled off and declined with the onset of the na- 
tional recession. This produced a very large earnings advantage for the Jobs-Plus group, which 
showed no signs of decaying over time. 

The graph in Panel B of Figure 4.8, for Southeast Asian men in St. Paul, shows that the 
baseline earnings trend of the Jobs-Plus group was somewhat below that of the comparison 
group and that both trends started almost at zero and climbed rapidly over time. Once again, 
however, after Jobs-Plus was implemented, earnings for the comparison group leveled off and 
declined somewhat with the onset of the recession, whereas those for the Jobs-Plus group con- 
tinued to climb rapidly. This very large earnings advantage for the Jobs-Plus group also showed 
no signs of decaying. 

The graphs in Figure 4.8 clearly illustrate the earnings histories that produced the impact 
estimates for men reported in Table 4.5. According to these findings, Jobs-Plus increased average 
annual earnings for Hispanic men in Los Angeles by $3,248 (or 28.3 percent) from 2000 to 2003. 
The corresponding increase for Southeast Asian men in St. Paul was $2,129 (or 20.6 percent). 
Thus, Jobs-Plus increased average earnings per person over four years by a total of $12,994 for 
Hispanic men in Los Angeles and $8,517 for Southeast Asian men in St. Paul. These findings fly 
in the face of the common perception that voluntary employment programs for economically dis- 
advantaged persons are not effective for men. 31 In addition, they provide clear evidence that such 



29 Eamings for these men are literally “off the charts” relative to other subgroups. Hence, it was necessary 
to extend the vertical axis in this figure to $4,000 (from a maximum value of $3,500 for all other subgroups). 
Figure E.4 presents corresponding graphs for quarterly employment rates. 

30 Baseline earnings for Hispanic men from Los Angeles were three to four times those for Hispanic 
women from the site. 

31 A prominent nonexperimental evaluation of the first federal job training program, the Manpower Devel- 
opment Training Act (MDTA), found positive earnings impacts for women but not for men (Ashenfelter, 
1978). However, a reanalysis of these data found positive earnings impacts for both groups (Bloom, 1984). An 
experimental evaluation of the National Supported Work Demonstration found positive earnings effects for 
women but not for men (MDRC, 1980). A major nonexperimental evaluation of the second federal job training 
program, the Comprehensive Employment and Training Act (CETA), found positive earnings impacts for 
women but not for men (Bloom and McLaughlin, 1982). Other researchers using the same data found mostly 
positive impacts for women but mixed effects for men (summarized by Barnow, 1987). An experimental 
evaluation of the third federal job training program, the Job Training Partnership Act (JTPA), found positive 
earnings impacts for men and women (Bloom et al., 1997). Most recently, a meta-analysis of the findings from 
31 evaluations of 15 voluntary employment programs operated between 1964 and 1998 found larger effects for 
women than for men (Greenberg, Michalopoulos, and Robbins, 2002). 
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The Jobs-Plus Demonstration 



Figure 4.8 



Average Quarterly Earnings for the Jobs-Plus Group and Its Comparison 
Group, for Men in the Largest Demographic Subgroups in the Stronger 
Implementation Sites (1998 Cohort) 



A. Los Angeles - Hispanic Men 




aaaaao'aaao'aa 

◄ Baseline Follow-Up ► 



B. St. Paul - Southeast Asian Men 
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SOURCES: MDRC calculations using data from housing authority tenant (50058) records and state 
Unemployment Insurance (UI) wage records. 

NOTE: The 1998 cohort includes all residents of a Jobs-Plus development or a comparison 
development in October 1998 who were between 21 and 61 years old and were not listed as disabled 
by their public housing authority. 
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programs can be effective for men from two different immigrant groups. Hence, they are highly 
relevant to decisions about future employment policies in the United States. 



The Impact of Jobs-Plus on Residents’ Welfare Receipt 

As noted earlier, Jobs-Plus was not a welfare -to-work program. Indeed, only between 
one-third and two-thirds of the public housing residents (depending on the site) were welfare 
recipients when Jobs-Plus was launched. Nevertheless, one of the principal factors motivating 
the Jobs-Plus demonstration was a concern over the potential implications of welfare reform, 
which was expected to reduce the ability of many public housing residents to pay their rent. 
Thus, welfare receipt was a key outcome of interest from the inception of the project. 

Figures 4.9, 4.10, and 4.11 present a graphical analysis of the effects of Jobs-Plus on 
welfare receipt for each of the three sites where data on this outcome were obtained: Dayton, 
Los Angeles, and St. Paul. The graph in Panel A of each figure illustrates how average quarterly 
benefits varied over time for the Jobs-Plus group and its comparison group. The graph in Panel 
B of each figure illustrates corresponding results for welfare receipt rates. On balance, these 
graphs indicate that although welfare receipt declined precipitously after Jobs-Plus was imple- 
mented, there is no evidence that the program caused this decline. 

Figure 4.9 indicates that welfare receipt for both the Jobs-Plus group and the compari- 
son group in Dayton remained roughly constant throughout the baseline period, followed by a 
sharp drop beginning in early 1998 and continuing thereafter. This pattern applies both to aver- 
age benefits received per quarter and to quarterly benefit receipt rates. Thus, for example, while 
roughly 50 percent of the Jobs-Plus group and the comparison group were receiving welfare 
during a given quarter in the baseline period, only about 10 percent were doing so by the end of 
the follow-up period. This dramatic reduction in welfare receipt mirrors similar patterns ob- 
served throughout the United States. 

There has been much debate about the extent to which a strong national economy, fed- 
eral welfare reform (with provisions such as time limits), changes in the culture of welfare, and 
other factors have caused these reductions in welfare receipt. And it is difficult, if not impossi- 
ble, to separate the effects of these forces. Nevertheless, it seems clear that Jobs-Plus did not 
cause this result in Dayton, because the pattern of welfare receipt over time is very similar for 
the Jobs-Plus group and its comparison group and because this pattern was virtually identical 
from about 1997 forward. 
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The Jobs-Plus Demonstration 
Figure 4.9 

Average Quarterly Welfare Benefits Received and Benefit Receipt Rates for the 
Jobs-Plus Group and Its Comparison Group in Dayton (1998 Cohort) 





SOURCES: MDRC calculations using data from housing authority tenant (50058) records and state 
AFDC/TANF records. 



NOTE: The 1998 cohort includes all residents of a Jobs-Plus development or a comparison 
development in October 1998 who were between 21 and 61 years old and were not listed as disabled 
by their public housing authority. 
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Figure 4.10 tells a similar story for Los Angeles, although some of its details are differ- 
ent. Welfare receipt for both the Jobs-Plus group and its comparison group remained roughly 
constant throughout much of the baseline period (after an initial aberration), followed by a sharp 
and sustained drop. 32 In this case, the drop started in early 1996, but the ultimate result was 
similar to that for Dayton. Quarterly welfare receipt rates fell from roughly 35 percent during 
the baseline period to just above 10 percent at the end of the follow-up period. Once again, there 
is no evidence that Jobs-Plus had an effect on this outcome. 

Figure 4.1 1 tells a similar story for St. Paul. However, this site had much bigger differ- 
ences between the Jobs-Plus group and the comparison group before, during, and after Jobs- 
Plus was launched. In addition, there was a sharp upward shift in welfare payments received 
during 1998 for both groups, without a corresponding increase in welfare receipt rates. This 
shift reflects a change in the structure of welfare payments that occurred when the Minnesota 
Family Investment Program (MFIP) was implemented statewide, allowing working recipients 
to keep more of their welfare grants. Nevertheless, there was an overall pattern of rapid decline 
in welfare receipt for both groups after this point in time. For example, benefit receipt rates 
dropped from a high of almost 70 percent in 1998 to a low of roughly 30 percent in 2003. 

In all three sites, the patterns of welfare benefits over time do not fit a readily specifi- 
able statistical model for estimating the effects of Jobs-Plus — although the graphs tell a pretty 
clear story. Thus, no statistical estimates are reported. Instead, the present analysis relies solely 
on an interpretation of the graphical findings. As noted, for Dayton and Los Angeles, these find- 
ings clearly demonstrate that Jobs-Plus had no effect on welfare receipt. The findings for St. 
Paul are more difficult to interpret because of time-varying baseline differences between the 
Jobs-Plus group and comparison group and a major change in the structure of welfare benefits 
when Jobs-Plus began. 



32 There is no clear explanation for the difference that exists in average benefits received by the Jobs-Plus 
group and comparison group during the first few quarters of the baseline period. This difference, which is 
much less pronounced for benefit receipt rates, may reflect early differences between the two groups in their 
family sizes or structures. 
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The Jobs-Plus Demonstration 



Figure 4.10 

Average Quarterly Welfare Benefits Received and Benefit Receipt Rates for the 
Jobs-Plus Group and Its Comparison Group in Los Angeles (1998 Cohort) 





SOURCES: MDRC calculations using data from housing authority tenant (50058) records and state 
AFDC/TANF records. 



NOTE: The 1998 cohort includes all residents of a Jobs-Plus development or a comparison 
development in October 1998 who were between 21 and 61 years old and were not listed as disabled 
by their public housing authority. 
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The Jobs-Plus Demonstration 
Figure 4.11 

Average Quarterly Welfare Benefits Received and Benefit Receipt Rates for the 
Jobs-Plus Group and Its Comparison Group in St. Paul (1998 Cohort) 





SOURCES: MDRC calculations using data from housing authority tenant (50058) records and state 
AFDC/TANF records. 

NOTE: The 1998 cohort includes all residents of a Jobs-Plus development or a comparison 
development in October 1998 who were between 21 and 61 years old and were not listed as disabled 
by their public housing authority. 
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Summary and Conclusions 

This chapter presents strong evidence that where and when Jobs-Plus was implemented 
relatively well it produced large earnings gains for public housing residents and proportionally 
smaller gains in employment rates. This “value added” by the program (particularly for earn- 
ings) is especially impressive for three reasons. First, it was sustained in the midst of a national 
economic recession, when jobs were becoming scarcer. Second, it followed almost a decade- 
long increase in earnings and employment fueled by a booming economy, which “raised the 
bar” with respect to what was required to produce further economic gains for program partici- 
pants. Third, it had to exceed the effects on comparison group members of other major changes 
in public policies that were designed to stimulate work among low-income persons. 

Although welfare receipt dropped precipitously after Jobs-Plus was launched, this de- 
cline was roughly the same for the Jobs-Plus group and comparison group. And, in some cases, 
the decline began before Jobs-Plus was implemented. Therefore, it is unlikely that Jobs-Plus 
had much effect on welfare receipt. Instead, it is more likely that factors such as the tail end of a 
booming economy, welfare reform, and increases in the generosity of the Earned Income Tax 
Credit caused the changes that were observed. 

All these findings are based on a strong evaluation design that gives them substantial 
credibility. Randomly choosing Jobs-Plus developments and comparison developments elimi- 
nates the potential for “stacking the deck” for or against the program through this selection 
process. 33 Grounding estimates of program impacts on many years of baseline data for the Jobs- 
Plus group and comparison group greatly enhances the ability to make fair comparisons of their 
future outcomes. And tracking outcomes for six years after the program was launched provides 
a substantial basis for gauging its ultimate success. This is especially important for complex, 
large-scale projects like Jobs-Plus that take a long time to implement and thus require the pas- 
sage of considerable time before their results can be seen. Further enhancing the credibility and 
utility of the findings is the fact that the patterns of program effects (both across sites and over 
time) closely match corresponding patterns in the program’s implementation — suggesting that 
the overall pattern of findings accords well with the theory of the program. 

The next logical step in evaluating Jobs-Plus is to consider how its impacts on individ- 
ual public housing residents (people) translate into impacts on public housing developments 
(places). Chapter 5 turns to this question. 



33 In econometric parlance, this means that “selection bias” was eliminated. 
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Chapter 5 



Work and Welfare Impacts 
on Public Housing Developments 

This chapter builds on Chapter 4’s analysis of the effects of Jobs-Plus on the earnings, 
employment, and welfare receipt of a specific cohort of public housing residents (some of 
whom moved away subsequently) to examine whether such effects on people get reflected by 
corresponding changes in their public housing developments [place ). The chapter thus seeks to 
answer the question: “How did Jobs-Plus affect the levels of earnings, employment, and welfare 
receipt that prevailed in the public housing developments where the program was conducted?” 
In other words, did Jobs-Plus help the developments become places where, at any given time, 
work was more common, tenants’ earnings were higher, and welfare receipt was lower than 
they would have been without the program? The chapter first outlines the basic issues involved 
and describes the data and analytic approach that were used. It then presents key findings from 
the analysis, which suggest that: 

• Not surprisingly, when no effects are produced for public housing residents, 
no effects are produced for the housing developments in which they live. 

• When positive effects are produced for residents, they are translated into 
positive effects for developments in inverse proportion to residents’ mobility: 

The lower the rate of resident mobility, the greater the degree to which indi- 
vidual effects are reflected by development effects. 



Basic Issues: Why Mobility Matters 

One way to think about the processes by which individual effects on public housing resi- 
dents might translate into neighborhood effects on public housing developments is to focus on 
three factors: individual program effects, individual mobility, and individual interactions. It seems 
reasonable to expect that, other things being equal, larger-than-average program impacts on indi- 
viduals will produce a greater-than-average impetus for change in their housing developments. It 
also seems reasonable to expect that the more frequently and intensively individual residents in- 
teract with each other, the greater the opportunity there is for program effects on some residents to 
“mb off’ on others, thereby magnifying the effects for their development. Furthermore, one might 
expect that high levels of resident mobility create a conduit though which large portions of the 
individual effects created by a program can be lost to a housing development. 
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Different combinations of these factors give rise to alternative scenarios about the po- 
tential relationships between individual program effects and development-level program effects. 
One scenario (“up and out”) specifies that residents are inherently mobile and that when their 
economic circumstances improve in response to a program, they move out of public housing — 
thereby taking their program benefits with them. Under this scenario, although public housing 
might be a good platform for providing services that help individual residents, these services 
might not improve conditions in public housing. 

A second scenario (“stability and synergy”) specifies that residents who participate in a 
program tend to stay in place — perhaps due to the program’ s economic incentives. Under this 
scenario, persons who benefit from a program keep their benefits within their development. 
Over time, as such individuals accumulate in the development, its conditions improve. And if 
residents interact with each other extensively, the success of some residents might breed further 
successes for others, creating synergy that multiplies the individual program effect to create an 
even larger development-level effect. 

A third scenario (“likes attract”) assumes that, as increased work becomes the norm for 
more individual residents, the development becomes a more desirable place for working-poor 
families to live. As word of this situation gets out, more such families might apply and be ac- 
cepted for residence in the development. 

With these scenarios in mind, one can create three different hypotheses for how devel- 
opment-level effects might play out in the Jobs-Plus study. First, for those developments in 
which there is no program effect on individual residents, there would be no expectation of a 
program effect at the development level. Second, for developments showing a moderate to large 
program effect on individual residents but an especially high rate of resident mobility, there 
would be a reduced expectation for a program effect at the development level. Finally, for those 
sites demonstrating a large program effect on individual residents and low levels of resident 
mobility, one would expect good prospects for large effects at the development level. 



Analytic Approach: Measuring Development Outcomes 
Over Time 

The first step in an analysis of the impacts of Jobs-Plus on public housing developments 
was to the identify all working-age, nondisabled residents in each development during every 
quarter from 1992 through 2003.' This information was obtained in various forms (electronic 



'To simplify the identification of residents in the sample during early baseline years (for which data from lo- 
cal public housing authorities were especially limited and difficult to obtain), only persons were included who 
lived in a household headed by a working-age, nondisabled person. Thus, a few able-bodied, working-age persons 

(continued) 
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and paper) from the administrative records and archives of local public housing authorities in 
the study (see Appendix D). From these records, it was possible to identify who was living in 
each development during each quarter of the baseline and follow-up periods for Jobs-Plus. 

The next step was to obtain historical data from Unemployment Insurance (UI) wage 
records and from welfare records (Aid to Families with Dependent Children [AFDC] and Tem- 
porary Assistance for Needy Families [TANF]) for every person who was identified as a resi- 
dent of the developments in the study at any time during the baseline or follow-up periods. This 
information was used to construct four quarterly outcomes for each Jobs-Plus development and 
its comparison development(s): average quarterly earnings (in 2003 dollars), quarterly employ- 
ment rates (in percentages), average quarterly welfare benefits received (in 2003 dollars), and 
quarterly benefit receipt rates (in percentages). As shown in Chapter 4, these outcomes were 
computed each quarter for members of the 1998 cohort or for members of the 2000 cohort — 
producing time-series data for the same individuals over time. For this chapter, these outcomes 
are computed each quarter only for persons who were residents of a Jobs-Plus development or a 
comparison development during that quarter. In other words, data for each person are only used 
for the quarters in which he or she was a resident of a Jobs-Plus development or a comparison 
development. These data produce a times series for the same places over time, with different 
persons living in these places at different times. 

The next step in the analysis was to construct graphs of the time paths of each outcome 
for each Jobs-Plus development and its comparison development(s). These graphs provide a 
visual analysis of how work or welfare outcomes in the developments compared before Jobs- 
Plus was launched and how this comparison changed (or not) after Jobs-Plus was launched. The 
final step in the analysis was to use the statistical model described in Appendix D to estimate 
the effects of Jobs-Plus from data in the graphs. These findings are presented below. 



The Impact of Jobs-Plus on Earnings and Employment in 
Developments 

Because the conditions for creating development-level program effects on work varied 
so much across sites, these outcomes are examined for each site separately. Figure 5 . 1 presents 
a site-by-site series of graphs that illustrate how average quarterly earnings changed over time 



who lived in a household headed by an elderly person or by a person listed as disabled by the public housing au- 
thority were omitted from the development-level impact analyses. 
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Mean Quarterly Earnings Mean Quarterly Earnings 

(in 2003 dollars) (in 2003 dollars) 



The Jobs-Plus Demonstration 



Figure 5.1 

Average Quarterly Earnings for the Jobs-Plus 
Group and Its Comparison Group, by Site (Development-Level Findings) 
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(continued) 
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Mean Quarterly Earnings Mean Quarterly Earnings 

(in 2003 dollars) (in 2003 dollars) 



Figure 5.1 (continued) 




D. Los Angeles 
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(continued) 
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Figure 5.1 (continued) 



E. St. Paul 




SOURCES: MDRC calculations using data from housing authority tenant (50058) records and state 
Unemployment Insurance (UI) wage records. 

NOTES: The development-level samples include all nondisabled residents aged 21 to 61 in each year. 
To facilitate data collection, given the complexity of local housing authority records, only persons 
living in a household headed by a nondisabled resident between the ages of 2 1 and 6 1 were included 
in these samples. 

Sample sizes varied from year to year, ranging from: 264 to 335 in the program group and 519 to 
666 in the comparison group in Baltimore; 240 to 314 in the program group and 349 to 500 in the 
comparison group in Chattanooga; 256 to 346 in the program group and 400 to 628 in the comparison 
group in Dayton; 356 to 466 in the program group and 294 to 387 in the comparison group in Los 
Angeles; and 192 to 289 in the program group and 149 to 21 1 in the comparison group in St. Paul. 

■“Unemployment Insurance (UI) wage data for the second quarter of 1993 were not available for 
Dayton. 



112 




for each Jobs-Plus development and comparison development(s). 2 Figure 5.2 presents a corre- 
sponding site-by-site series of bar charts that illustrate the annual effects of Jobs-Plus on devel- 
opment-level earnings. 3 And Table 5.1 compares estimated development-level impacts on earn- 
ings for the period 2000-2003 with their individual-level counterparts for each site. Seattle is not 
included because the HOPE VI renovation at its Jobs-Plus development appears to have influ- 
enced the composition of residents in ways that would invalidate a development-level analysis 
of the effects of Jobs-Plus. 4 

It is important to note that the development-level findings reflect more random period- 
to-period fluctuation than do corresponding individual-level findings from the preceding chap- 
ter. This is because development-level analyses contain an added source of random error: that 
which is due to changes in the composition of the sample over time as some residents move out 
and others move in. Hence, development-level estimates of program effects are less precise than 
are individual-level estimates. Now, consider the findings site by site. 

Baltimore Had No Development-Level Impacts 

Chapter 4 reports that Baltimore had far less program penetration than any other site 
and was one of only two sites with no program effects on individual work outcomes. Thus, one 
would expect no development-level effects on work at this site. This expectation is borne out by 
the data for Baltimore in Figure 5.1. Panel A illustrates that average quarterly earnings in Balti- 
more were similar for the Jobs-Plus development and comparison developments during the six- 
year baseline period and for the first three years of follow-up. Although average quarterly earn- 
ings for both sets of developments roughly doubled during this time, they remained very close 
to each other. For the next three years, however, earnings in the Jobs-Plus development dropped 
somewhat, and earnings in the comparison developments leveled off, as the national economic 
recession took effect. The Jobs-Plus development appears to have done less well than did the 
comparison developments during the recession, although this gap closed by the end of 2003. 

Figure 5.2 presents estimates of the effects of Jobs-Plus on development-level earnings 
based on the data in Figure 5.1. For Baltimore, these findings indicate negligible program effects 
on earnings from 1998 to 2000 followed by negative effects from 2001 through 2003, although 

“Appendix Figure F.l presents corresponding graphs for quarterly employment rates. 

^Because of an early, temporary discrepancy between earnings for the Jobs-Plus development and compari- 
son development in Los Angeles (see Panel D of Figure 5.1), all estimates of program impacts on developments 
presented in this chapter use a truncated baseline period that begins in 1994. The same baseline period is used for 
all sites, to make their findings comparable. Appendix D presents estimates using the fall baseline period, which 
are similar to those presented in this chapter except for Los Angeles, where they are considerably larger. 

4 The data indicate that HOPE VI in Seattle differentially affected the mobility of employed and nonem- 
ployed residents of the Jobs-Plus development, which is consistent with other information obtained from field 
research at the site. 
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The Jobs-Plus Demonstration 



Figure 5.2 



Average Annual Effects of Jobs-Plus on Earnings, by Site 
(Development-Level Findings) 



Baltimore 



Dayton 



1998 


1999 


2000 


2001 


2002 


$21 




$66 

(1.0%) 






1 

(-0.4%) 


-$79 

(-1.4%) 




(-9.1%) 


-$848 ** 
(-14.0%) 


$73 

(1.3%) 




$338 

(4.5%) 


$704 

(10.4) 


$855 

(13.9%) 









-$240 



2003 



-$599 

(-9.6%) 



$438 

(7.2%) 



(-3.4%) 



Los Angeles 



$437 $483 

(7.1%) (7.0%) 



$1,415** $1,757*** 

(20.2%) (26.2%) 




$1,534*** $1,619*** 
(22.3%) (21.9%) 




St. Paul 



$436 

(6.7%) 



$1,751 

(24.4%) 




$1,290* 

(15.7%) 



$1,718** 

(20.4%) 




$1,279 ** 
(12.5%) 



$1,247** 

(11.4%) 



(continued) 

SOURCES: MDRC calculations using data from housing authority tenant (50058) records and state 
Unemployment Insurance (UI) wage records. 

NOTES: All earnings effects are reported in 2003 dollars. 

The development-level samples include all nondisabled residents aged 21 to 61 in each year. To 
facilitate data collection, given the complexity of local housing authority records, only persons living in 
a household headed by a nondisabled resident between the ages of 21 and 61 were included in these 
samples. 

Statistical significance levels are indicated as *** = 1 percent; ** = 5 percent; * = 10 percent. 
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Figure 5.2 (continued) 



Sample sizes varied from year to year, ranging from: 264 to 335 in the program group and 519 to 
666 in the comparison group in Baltimore; 240 to 314 in the program group and 349 to 500 in the 
comparison group in Chattanooga; 256 to 346 in the program group and 400 to 628 in the comparison 
group in Dayton; 356 to 466 in the program group and 294 to 387 in the comparison group in Los 
Angeles; and 192 to 289 in the program group and 149 to 21 1 in the comparison group in St. Paul. 

Estimates of the impacts of Jobs-Plus on development-level earnings are not reported for 
Chattanooga because of the ambiguous relationship between the baseline earnings patterns of its Jobs- 
Plus and comparison developments. 



only the finding for 2002 is statistically significant. Given that Jobs-Plus was implemented weakly 
in Baltimore and that it produced no earnings effects for individual residents, the most plausible 
interpretation of the findings is that the program produced no development-level effects in Balti- 
more. The short-lived development-level earnings loss that was observed probably represents a 
temporary aberration or population changes that had nothing to do with Jobs-Plus. 

Chattanooga’s Development-Level Impacts Were Difficult to Estimate 

Chattanooga had the second-lowest rate of Jobs-Plus participation as indicated by take- 
up of the rent-based work incentives; its program devolved to rent incentives only; and it did not 
increase the earnings of individual public housing residents. Hence, there is no reason to expect 
program impacts on earnings at the development level. Figure 5.1, Panel B, provides ambiguous 
findings with respect to this effect. 

During the baseline period for which data are available (1994 to 1997), earnings for the 
Jobs-Plus development were initially much lower than those for the comparison development. 
However, earnings for the Jobs-Plus development rose much more quickly than those for the 
comparison development so that they were almost identical when Jobs-Plus was launched in 
1998. Subsequently, earnings for both developments fluctuated from quarter to quarter but did 
not change appreciably, on average. 

If one assumes that earnings for the two developments would have been similar without 
Jobs-Plus (because they were virtually identical when the program began), then one would con- 
clude from the data that Jobs-Plus had no effect on development earnings. If, instead, one as- 
sumes that the baseline earnings trajectories for the two sets of developments (which intersect at 
the point when Jobs-Plus was launched) would have continued without Jobs-Plus, then the data 
suggest that Jobs-Plus actually reduced future earnings for its development. Given that Jobs- 
Plus in Chattanooga was ultimately limited to rent incentives only, had relatively low rates of 
participation, and had no effect on individual earnings, it seems most plausible to conclude that 



115 




The Jobs-Plus Demonstration 
Table 5.1 



Comparison of Individual-Level and Development-Level Effects of Jobs-Plus on 
Average Annual Earnings from 2000 to 2003, by Site 



Site 


Earnings Effect for 1998 Cohort 


Earnings Effect for Jobs-Plus Development 


Dollars 


Percentage 


Dollars 


Percentage 


Baltimore 


-189 


-2.7 


-489 ** 


-7.8 


Dayton 


895 *** 


13.0 


584 


8.8 


Los Angeles 


1,120 *** 


14.8 


1,581 *** 


22.6 


St. Paul 


1 492 *** 


15.4 


1,384 *** 


14.6 



SOURCES: MDRC calculations using data from housing authority tenant (50058) records and state 
Unemployment Insurance (UI) wage records. 

NOTES: All earnings effects are reported in 2003 dollars. 

The 1998 cohort includes all residents of a Jobs-Plus development or a comparison development in 
October 1998 who were between 21 and 61 years old and were not listed as disabled by their public housing 
authority. 

The development-level samples include all nondisabled residents aged 21 to 61 in each year. To 
facilitate data collection, given the complexity of local housing authority records, only persons living in a 
household headed by a nondisabled resident between the ages of 21 and 61 were included in these samples. 

Statistical significance levels are indicated as *** = 1 percent; ** = 5 percent; and * = 10 percent. 

Sample sizes varied from year to year, ranging from: 264 to 335 in the program group and 519 to 666 
in the comparison group in Baltimore; 256 to 346 in the program group and 400 to 628 in the comparison 
group in Dayton; 356 to 466 in the program group and 294 to 387 in the comparison group in Los Angeles; 
and 192 to 289 in the program group and 149 to 21 1 in the comparison group in St. Paul. 

Estimates of the impacts of Jobs-Plus on development-level earnings are not reported for Chattanooga 
because of the ambiguous relationship between the baseline earnings patterns of its Jobs-Plus and 
comparison developments. 



the program had no development-level effect on earnings. Nevertheless, because of the ambigu- 
ity created by the poor and highly variable baseline match in earnings, no estimates are pre- 
sented for the development-level effects of Jobs-Plus in this site. 
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Dayton Had Moderate Development-Level Impacts 

Dayton had the highest rate of resident mobility (with 48 percent of its 1998 cohort 
members moving away within two years) and moderate-to-large program impacts on individual 
residents’ earnings. One might then expect the development-level effects of Jobs-Plus in this 
site to be a diminished version of the effects for individual residents. 

This expectation is borne out by Panel C in Figure 5.1. First, note that average earnings 
for the Jobs-Plus development and the comparison developments in Dayton increased between 
threefold and fourfold from 1992 to 2000. Thereafter, earnings for both groups declined with 
the onset of the recession. This decline was more pronounced for the comparison development, 
however, and thus the Jobs-Plus development exhibited an earnings advantage from 2000 for- 
ward. The fact that this earnings advantage is larger than that for the baseline period suggests 
that Jobs-Plus increased development-level earnings. 

Figure 5.2 shows that Jobs-Plus’ s effects on development-level earnings for Dayton are 
positive but smaller than those for individual residents (see Chapter 4, Figure 4.4). In addition, 
the estimated development-level program effects for each year are not statistically significant. It 
appears then that the individual-level effects of Jobs-Plus in Dayton were translated into devel- 
opment-level effects that were somewhat less pronounced. 

Los Angeles Had Large Development-Level Impacts 

Resident mobility in Los Angeles was considerably lower than that for any other site 
(with only 17 percent of its 1998 cohort members moving away within two years). Furthermore, 
this site experienced one of the largest impacts of Jobs-Plus on individual earnings. Thus, one 
would expect it to have good prospects for development-level effects. 

Once again, the analysis bears out the expectation. Panel D of Figure 5.1 illustrates 
that earnings for the Jobs-Plus development and the comparison development in Los Angeles 
increased dramatically from 1992 to 2000, followed by a slowdown as the national economic 
recession set in. The slowdown was much greater for the comparison development than for 
the Jobs-Plus development, however, which produced a substantial earnings advantage for the 
Jobs-Plus development. The fact that this earnings advantage was much greater than that dur- 
ing the baseline period suggests a large positive effect of Jobs-Plus on development-level 
earnings in Los Angeles. 

Correspondingly, Figure 5.2 indicates that the effects of Jobs-Plus on development- 
level earnings in Los Angeles are consistently large and statistically significant. It appears that 
the large positive effects of Jobs-Plus on the work levels of public housing residents in this site 
translated into large positive effects on levels of work in their public housing development. 
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St. Paul Had Large Development-Level Impacts 

St. Paul also had a low level of resident mobility (with only 27 percent of its 1998 cohort 
members moving away within two years) and large Jobs-Plus impacts on individual earnings. 
Thus, it too had good prospects for program-induced earnings gains at the development level. 

That this occurred can be seen from the findings shown in Panel E of Figure 5.1. First, 
note that earnings more than tripled from 1994 (when Minnesota’s UI wage records became 
available) to 1998 for the Jobs-Plus development and its comparison development. In addition, 
the baseline match for these developments was quite good, meaning that future comparisons 
between them should be valid. Furthermore, unlike the pattern for other sites, average earnings 
continued to increase for both developments during the national recession. However, the rate of 
this increase was much greater for the Jobs-Plus development after 1999, when the program’s 
employment services and rent incentives had been implemented. Thus, a clear and consistent 
earnings advantage was experienced by the Jobs-Plus development from this time forward. 

Consistent with this pattern of findings, Figure 5.2 presents a series of large and statisti- 
cally significant estimated effects of Jobs-Plus on development earnings in St. Paul, beginning 
in 1999. It seems clear that the program’s effect on individual earnings was translated into a de- 
velopment-level effect. 

Resident Mobility Reduced Development-Level Impacts 

Table 5.1 summarizes the preceding development-level findings and compares them 
with their individual-level counterparts for the 1998 cohort. To do so, the table presents esti- 
mates of the effects of Jobs-Plus on average annual earnings between 2000 and 2003 from both 
the individual and the development perspectives. These estimates are reported in 2003 dollars 
and as a percentage of what earnings would have been without the program. 

For Dayton, Fos Angeles, and St. Paul, there is an intriguing relationship between rates 
of resident mobility and the degree to which individual program effects translated into devel- 
opment impacts. In Dayton, where 48 percent of the 1998 cohort members moved away within 
two years, roughly two-thirds of the individual effect of Jobs-Plus on earnings were translated 
into a development effect. In St. Paul, where 27 percent of the 1998 cohort members moved 
away within two years, almost all of the individual effect of Jobs-Plus on earnings was trans- 
lated into a development effect. In Fos Angeles, where only 17 percent of the 1998 cohort 
members moved away within two years, the effect of Jobs-Plus on development earnings was 
40 percent larger than the individual effect of the program. These three sites present evidence 
that resident mobility reduces the extent to which the individual benefits of a place-based em- 
ployment program translate into benefits for the community. Of course, given the number of 



118 




factors that change from period to period at each development and the amount of random error 
that exists in the data, these findings should be considered suggestive rather than definitive. 

Employment Rates for Developments Moved with the Economy 

Because Jobs-Plus did not exhibit a clear pattern of individual-level effects on employ- 
ment rates, a development-level analysis of program effects is not presented for this outcome. 
However, because of the dramatic changes over time in employment rates that occurred, it is 
still useful to describe these outcomes for public housing developments in the study. Table 5.2 
does so by listing the employment levels for each Jobs-Plus development four years before the 
program was launched (Quarter 3 of 1994), while it was being launched (Quarter 3 of 1998), 
and five years after it was launched (Quarter 3 of 2003). These rates are for residents of the 
Jobs-Plus development in each site who (at the time) were between 21 and 61 years old and 
were not listed as disabled in records of their public housing authority. Therefore, findings for 
each point in time represent the experiences of the same places but different people. 

Employment rates increased dramatically at all Jobs-Plus developments during the four 
years before the program was launched, probably due to the unusually strong and sustained 
economic progress being made nationally. As noted earlier, this type of finding provides strong 
evidence that when jobs are available, residents of public housing will seek them out and take 
them. Because of this, employment rates that were in effect when Jobs-Plus got under way were 
much higher than those indicated by the data used to choose the sites. These high baseline rates 
of employment came as a surprise to planners of the demonstration program — and may have 
reduced (though did not eliminate) the margin for it to create further improvement. 

As Jobs-Plus got under way and the national economic boom became a national eco- 
nomic recession, employment rates in the Baltimore and Dayton Jobs-Plus developments 
stopped rising. Hence, the rates were almost exactly the same five years later. In contrast, the 
development-level employment rates in Los Angeles and St. Paul continued to rise, although at 
a noticeably slower pace than previously. In no site, however, did earnings or employment 
reach a tipping point, setting off an acceleration in the pace of change in employment levels 
within public housing. 



The Impact of Jobs-Plus on Welfare in Developments 

This section briefly examines the effects of Jobs-Plus on welfare receipt in public hous- 
ing developments from the three sites for which data on this outcome were obtained: Dayton, 
Los Angeles, and St. Paul. Given that no program effects on welfare were observed for indi- 
viduals at these sites, no welfare effects on housing developments were expected. However, 
examining the patterns of welfare receipt over time in these sites can give a sense of how they 
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The Jobs-Plus Demonstration 
Table 5.2 



Quarterly Employment Rates for Working-Age, Nondisabled Residents 
at Each Jobs-Plus Development 



Site 




Percentage Employed 




Quarter 3 1994 


Quarter 3 1998 


Quarter 3 2003 


Baltimore 


27 


42 


43 


Chattanooga 


52 


70 


52 


Dayton 


44 


61 


62 


Los Angeles 


35 


46 


53 


St. Paul 


24 


52 


58 



SOURCES: MDRC calculations using data from housing authority tenant (50058) 
records and state Unemployment Insurance (UI) wage records. 

NOTES: The development-level samples include all nondisabled residents aged 21 to 61 
in each year. To facilitate data collection, given the complexity of local housing authority 
records, only persons living in a household headed by a nondisabled resident between the 
ages of 21 and 61 were included in these samples. 

Sample sizes varied from year to year, ranging from: 264 to 335 in the program 
group and 519 to 666 in the comparison group in Baltimore; 240 to 314 in the program 

group and 349 to 500 in the comparison group in Chattanooga; 256 to 346 in the program 

group and 400 to 628 in the comparison group in Dayton; 356 to 466 in the program 

group and 294 to 387 in the comparison group in Los Angeles; and 192 to 289 in the 

program group and 149 to 21 1 in the comparison group in St. Paul. 



changed and of the likely forces that caused these changes to occur. Indeed, some of the find- 
ings observed for the Jobs-Plus sites provide evidence about what is likely to be driving the de- 
clines in welfare caseloads that have been experienced nationally. 

Welfare Payments Declined Precipitously, but Jobs-Plus Had No Impact 

Figure 5.3 presents time-series graphs of average quarterly AFDC/TANF payments re- 
ceived by working-age, nondisabled residents of each Jobs-Plus development and its compari- 
son development! s) in the three sites. The most striking finding from these graphs is the dra- 
matic decline in welfare benefits payments that occurred in both the Jobs-Plus developments 
and their comparison developments starting in 1997 at Dayton and Los Angeles and in 1998 at 
St. Paul. These precipitous declines mirror those observed by the individual-level analysis in 
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Mean Quarterly Benefits Received Mean Quarterly Benefits Received 

(in 2003 dollars) (in 2003 dollars) 



The Jobs-Plus Demonstration 



Figure 5.3 



Average Quarterly Welfare Benefits Received for the Jobs-Plus Group 
and Its Comparison Group, by Site (Development-Level Findings) 
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Figure 5.3 (continued) 



C. St. Paul 




SOURCES: MDRC calculations using data from housing authority tenant (50058) records and state 
AFDC/TANF records. 

NOTES: The development-level samples include all nondisabled residents aged 21 to 61 in each year. 
To facilitate data collection, given the complexity of local housing authority records, only persons 
living in a household headed by a nondisabled resident between the ages of 2 1 and 6 1 were included in 
these samples. 

Sample sizes varied from year to year, ranging from: 256 to 346 in the program group and 400 to 
628 in the comparison group in Dayton; 356 to 466 in the program group and 294 to 387 in the 
comparison group in Los Angeles; and 192 to 289 in the program group and 149 to 21 1 in the 
comparison group in St. Paul. 



Chapter 4 and experienced by welfare systems throughout the United States. As noted earlier, 
there is considerable debate about what caused this decline — including hypotheses about the 
relative roles of the booming U.S. economy, welfare reform (which includes among its many 
provisions time limits on welfare receipt), and changes in the culture of welfare — but there is 
no debate that the decline has been sharp and sustained. 
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As for evidence about the effects of Jobs-Plus on welfare benefits payments, the graphs 
are more difficult to interpret than are their counterparts for earnings. Nevertheless, the overall 
impression they convey is one of no apparent Jobs-Plus effects. 5 For example, welfare benefits 
payments in Dayton were slightly lower for the Jobs-Plus development from about 1992 to 
1997 and then almost identical for the Jobs-Plus and comparison developments thereafter. 
Hence, whatever caused benefits to be the same for the two developments occurred before Jobs- 
Plus was launched. 

In Los Angeles, there was an initial large discrepancy between the levels of welfare 
benefits payments received by the Jobs-Plus development and comparison development. (This 
discrepancy is the mirror image of that observed for earnings in Figure 5.1.) However, by 1996, 
the discrepancy had disappeared, and the level of welfare benefits received by the Jobs-Plus 
development was slightly less than the level received by the comparison development. This re- 
lationship was sustained for a number of years thereafter and then diminished as the overall 
level of welfare benefits payments for both developments declined to a small fraction of their 
initial levels. Once again, there is little or no evidence that Jobs-Plus had an effect on welfare 
benefits received. 

For St. Paul, the findings are more difficult to interpret. Throughout the baseline pe- 
riod, average welfare payments for the Jobs-Plus development were much higher than those 
for the comparison development. Then, in 1998, when Jobs-Plus was launched, average pay- 
ments for both groups rose abruptly as the state's new welfare program, the Minnesota Fam- 
ily Investment Program (MFIP), was implemented statewide. This program changed major 
provisions of the welfare system in Minnesota, including its benefit payment structure. Sub- 
sequently, welfare payments declined sharply for both the Jobs-Plus development and its 
comparison development. By the end of the follow-up period, benefit levels were so low for 
both groups that there was little margin for a difference. To interpret this finding, it is impor- 
tant to also examine corresponding trends in welfare receipt rates for the two developments. 
Findings in Appendix Figure F.2 demonstrate that receipt rates were almost identical 
throughout the baseline and follow-up periods. Hence, there is no sign of a program effect on 
welfare benefits for the Jobs-Plus development in St. Paul. 



5 Because the complex patterns represented by the time series for welfare benefits in Figure 5.3 do not fit 
an easily identifiable statistical model, numeric estimates of the impacts of Jobs-Plus on welfare receipt are not 
presented. Nevertheless, the basic conclusion to be drawn from the data is that Jobs-Plus did not affect this 
outcome for public housing developments, as was also the case for public housing residents. 
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Welfare Receipt Rates Did Not Always Move with Employment Rates 

To complete the present analysis, Table 5.3 lists welfare receipt rates for each Jobs-Plus 
development in the three sites during Quarter 3 of 1994, 1998, and 2003. First, note the findings 
for 1994 to 1998, before Jobs-Plus was launched. As can be seen, welfare receipt rates dropped 
dramatically in all three sites during this four-year period. These fast-falling welfare receipt 
rates mirror the rapidly rising employment rates in Table 5.2. 



The Jobs-Plus Demonstration 
Table 5.3 



Quarterly Welfare Receipt Rates for Working-Age, Nondisabled 
Residents at the Jobs-Plus Developments, by Site 



Site 


Percentage Receiving AFDC/TANF 


Quarter 3 1994 


Quarter 3 1998 


Quarter 3 2003 


Dayton 


66 


44 


24 


Los Angeles 


41 


28 


17 


St. Paul 


74 


65 


26 



SOURCES: MDRC calculations using data from housing authority tenant (50058) 
records and state AFDC/TANF records. 

NOTES: The development-level samples include all nondisabled residents aged 21 to 61 
in each year. To facilitate data collection, given the complexity of local housing authority 
records, only persons living in a household headed by a nondisabled resident between the 
ages of 21 and 61 were included in these samples. 

Sample sizes varied from year to year, ranging from: 256 to 346 in the program 
group and 400 to 628 in the comparison group in Dayton; 356 to 466 in the program 
group and 294 to 387 in the comparison group in Los Angeles; and 192 to 289 in the 
program group and 149 to 21 1 in the comparison group in St. Paul. 



But consider what happened subsequently. Unlike employment rates, which stopped 
rising altogether (in Dayton) or continued to rise but more slowly (in Los Angeles and St. Paul), 
welfare receipt rates continued to plummet from 1998 to 2003. As noted above, there is no evi- 
dence that this phenomenon has anything to do with Jobs-Plus. However, its occurrence sheds 
some interesting light on the likely role played by economic conditions on declining welfare 
rolls. Specifically, during the period from 1994 to 1998, while the national economy was still 
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hot, employment rates in the three Jobs-Plus developments rose, and welfare receipt rates fell. 
This is consistent with the hypothesis that a strong economy reduced welfare receipt (but, as 
noted, many other forces were at work as well). However, during the period from 1998 to 2003 
— which was marked, in part, by a national recession — employment rates stood still or rose 
slowly, but welfare receipt rates continued to free-fall. Therefore, one must look beyond the 
economy for an explanation of the later continuing decline in the welfare rolls. 



Summary and Conclusions 

This chapter demonstrates a consistent relationship between the mobility of public 
housing residents and the degree to which individual benefits from a placed-based employment 
program are translated into corresponding benefits for public housing developments: The less 
the mobility, the greater the degree to which individual program benefits translate into benefits 
for developments. Thus, the findings illustrate one important way that mobility mediates the 
linkages between people and place. A next step toward better understanding this complex set of 
relationships is to consider how changes in employment and welfare receipt in public housing 
developments do or do not get reflected in other important community outcomes. This issue is 
explored in Chapter 6. 
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Chapter 6 



Jobs-Plus and Changes in the Quality of Life 
at Public Housing Developments 

The preceding chapters show that Jobs-Plus operated mostly in an environment of ris- 
ing earnings and employment, even for very economically disadvantaged populations. When 
the program was implemented well, it was able to improve work outcomes further for many 
participants. This chapter considers ways in which the growing climate of work among public 
housing residents might have changed the general quality of life in their housing developments, 
according to such measures as economic and material well-being, personal safety, residential 
satisfaction, and child well-being. 

The chapter addresses the following questions: 

• How did the Jobs-Plus developments change between the year the program 
began (1998) and the year it ended (2003) on measures of quality of commu- 
nity life and resident well-being? 

• How did these changes match up against changes on the same quality-of-life 
measures in comparison developments? 

• Is there evidence that positive effects of Jobs-Plus on residents’ work trans- 
lated into quality-of-life improvements for them and their public housing de- 
velopments? 

In general, the findings show that little change occurred in quality-of-life outcomes for resi- 
dents of Jobs-Plus developments between the baseline survey and the follow-up survey. Where 
changes did occur, they usually were no different from those observed for comparison develop- 
ments. Thus, even though Jobs-Plus increased earnings appreciably when it was implemented well, 
these changes were not large enough to “transform” the quality of life in public housing. 



Expectations for Community Change 

The theoretical framework underlying Jobs-Plus builds on work by William Julius Wil- 
son and others who view many of today’s problems in high-poverty neighborhoods — crime, 
welfare dependence, family dissolution, and social disorganization — as fundamental conse- 
quences of the disappearance of work. Wilson argues that these problems cannot be solved 
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without substantially raising the proportion of residents in such neighborhoods who are gain- 
fully employed. 1 Implicit in this argument is the hypothesis that dramatic increases in employ- 
ment will bring wider improvements in neighborhood quality of life. For reasons explained be- 
low, the Jobs-Plus evaluation does not provide a rigorous test of this hypothesis. However, the 
rich survey data collected for the study make it possible to explore how Jobs-Plus developments 
changed along important dimensions of community life while the program was being operated. 
In addition, these data allow one to compare patterns of change for the Jobs-Plus developments 
with those for comparison developments. Such quantifiable evidence on well-being and quality- 
of-life measures within public housing is rare. 

This analysis relies on resident surveys conducted at the start of Jobs-Plus in 1998 and 
five years later at the three study sites (Baltimore, Dayton, and St. Paul) where the full version 
of the 2003 follow-up survey was administered. 2 This chapter examines changes in several ma- 
jor categories of quality-of-life outcomes: economic and material well-being, social capital, per- 
sonal safety and victimization, neighborhood conditions and residential satisfaction, and child 
well-being. When considering these findings, it is important to keep in mind the different results 
for each site, which are explained in detail in earlier chapters: 

• Baltimore. Jobs-Plus in this site faced a host of implementation and opera- 
tional problems that compromised the quality of the program and may have 
contributed to its lack of effects on work outcomes at both the individual and 
the development levels. 

• Dayton. Characterized by a fairly strong and well-implemented intervention, 
Jobs-Plus in this site registered modest to large effects on individual-level 
earnings, which were muted at the development level because of high rates 
of resident mobility. 

• St. Paul. This site’s Jobs-Plus program was strong and well implemented, 
and it produced substantial effects on individual and development earnings. 

In addition, fewer residents moved away, thus potentially allowing for spill- 
over effects on other aspects of community life. 

These three different sites with three different Jobs-Plus experiences are used to ex- 
plore potential links between changes in work behavior and changes in quality of life within 
public housing. 



'Wilson, 1996. 

“Due to cost considerations, a reduced version of the survey was administered to smaller samples in Chat- 
tanooga and Los Angeles. See Appendix C for additional details on the Jobs-Plus survey. 
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Cautions Regarding These Findings 

Readers should be cautious when interpreting the results of this analysis of potential 
spillover effects, for at least three reasons: the level of change in employment and earnings ex- 
perienced between the surveys is not large enough to provide a strong test of community spill- 
over effects; the research method adopted is not as rigorous as the one used for the analyses in 
Chapters 4 and 5; and this study is not equipped to analyze the pathways through which any 
spillover effects might have occurred. Consider each of these reasons in turn. 

First, the change in employment and earnings that occurred during the survey analysis 
period (1998-2003) was not dramatic enough to support a strong test of spillover effects. Table 
6.1 illustrates that although there were very large increases in employment and earnings before 
Jobs-Plus began (1994-1998), changes in these measures were much smaller while the program 
was under way (1998-2003). For example, note the dramatic jump in employment rates be- 
tween 1994 and 1998: 15 percentage points in Baltimore, 16 percentage points in Dayton, and 
27 percentage points in St. Paul. In stark contrast, employment rates (and earnings) in Baltimore 
and Dayton rose only very slightly from 1998 to 2003. Hence, one would not expect to observe 
large changes in quality of life in these sites during that period. At the third site, St. Paul, em- 
ployment rates rose by 7 percentage points from 1998 to 2003, and average earnings continued 
to rise substantially, which would lead one to expect at least the possibility of spillover effects 
on quality-of-life outcomes. But even in St. Paul, the increased amount of work produced by 
Jobs-Plus probably did not “transform” the housing development into a low-poverty neighbor- 
hood, so the ability to link changes in work to improvements in quality of life is limited. 

Second, compared with the interrupted time-series analysis used in Chapters 4 and 5 to 
measure program effects on work and welfare, the analyses presented here are based only on 
two cross-sectional surveys that were administered to tenants of the developments at two points 
in time. For the most part, because of resident mobility, different people participated in each 
survey (see Appendix Table C.l). The absence of long-term trend data on quality-of-life out- 
comes for residents of these developments limits the ability to attribute changes in community 
outcomes to Jobs-Plus. Readers should be careful not to draw causal conclusions about the ef- 
fects of Jobs-Plus on quality of life. 

Third, this study cannot adequately measure the pathways through which spillovers 
might occur within public housing communities. For example, how do spillover mechanisms 
operate in highly dynamic settings? What types of interactions create a positive environment for 
promoting spillovers? Among whom? If a critical pathway for achieving positive spillovers is 
the social network of residents (that is, the people they know in the community and their ties to 
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The Jobs-Plus Demonstration 
Table 6.1 



Quarterly Earnings and Employment Rates for the Jobs-Plus Developments 

at Key Points in Time 



Sites 


1994 


1998 


2003 


Average earnings in Ouarter 3 ($) 

Baltimore 


792 


1,252 


1,357 


Dayton 


992 


1,403 


1,606 


St. Paul 


759 


1,838 


2,998 


Employed in Ouarter 3 (%) 

Baltimore 


27.3 


41.8 


43.2 


Dayton 


44.4 


60.8 


62.1 


St. Paul 


24.5 


51.9 


58.5 



SOURCE: MDRC calculations using data from state Unemployment Insurance (UI) wage records. 



NOTES: All earnings are reported in 2003 dollars. 

neighbors), then resident stability is an important condition for realizing the kind of neighbor- 
hood turnaround envisioned by Jobs-Plus. To the extent that public housing developments ex- 
perience a significant degree of residential turnover (as in Dayton, for example), then such mo- 
bility may prevent spillover effects from taking root . 3 Further, if successful program participants 
move away and lose their ties to the neighborhood, then the neighborhood loses the very people 
needed to support its improvement. 



3 The implicit theory behind a place-based, self-sufficiency initiative like Jobs-Plus assumes that the fol- 
lowing conditions, at a minimum, would need to be met for a program that is effective for individuals to have 
spillover effects on their immediate community: (T) a significant portion of the eligible target-area residents 
will be exposed to the program; (2) the eligible residents will stay long enough in the target area to be exposed 
to all aspects of the program for a significant amount of time; (3) program participants and beneficiaries will 
stay in the target area long enough to influence neighbors and others in the community; and (4) in the event that 
target-area residents move out of their original neighborhoods, they will maintain close connections with their 
former neighbors to continue to provide access to employment information and opportunities. 



130 







Measuring Quality-of-Life Changes 



The Samples 

Baseline and follow-up surveys for Jobs-Plus were administered to household heads 
who were (1) living in a Jobs-Plus or comparison development at the time of sample selection 
and survey administration, (2) not identified by the public housing authority as being disabled, 
and (3) between 21 and 61 years old at the time of sample selection. 4 In Baltimore and Dayton, 
the baseline survey was conducted in the spring and summer of 1998 and provides a snapshot of 
the people and developments when Jobs-Plus began; the survey for St. Paul was fielded in 
1999, roughly one year after the baseline survey was fielded in the other sites. This chapter re- 
fers to this survey as the baseline or the 1998 survey. 

A more comprehensive follow-up survey was fielded in three of the six demonstration 
sites. Thus, these three sites — Baltimore, Dayton, and St. Paul — are the focus of the commu- 
nity change analysis. Only about 30 percent of the sample responded to both the baseline and 
the follow-up survey. Not surprisingly, given the differences in resident move-out rates, such 
overlap was highest in St. Paul (39 percent) and lowest in Dayton (19 percent); Baltimore was 
in the middle with 29 percent. 

A total of 611 residents from the Jobs-Plus developments and 621 residents from the 
comparison developments responded to the baseline survey. Another 513 residents from the 
Jobs-Plus developments and 689 residents from the comparison developments responded to the 
follow-up survey. Response rates for the two surveys were over 80 percent. Estimates of 
changes in community outcomes were based on differences in responses to the two surveys. 
Appendix C provides more information about the surveys and samples used. 

Data on child well-being — in the form of parental responses to survey questions about 
their children — were collected for children of respondents in two of the three sites: Baltimore and 
Dayton. A total of 817 children ages 6 to 17 were included in the baseline analysis; another 777 
children were included in the follow-up analysis. In both sites, there are more children in the 6-to- 
1 1 age range than in the 12-to-17 age group. As a result, the sample for examining outcomes for 
older children is smaller than the sample for examining outcomes for younger children. Given 
these smaller samples, the survey findings on child outcomes should be interpreted cautiously. 



4 This definition is consistent with the one used to define cohorts for the impact analyses presented in 
Chapters 4 and 5. 
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The Approach to Assessing Change 

The analysis proceeded in two stages. The first stage focused on the question “Did the 
Jobs-Plus developments change over time in any appreciable way on selected measures of qual- 
ity of life?” Here the emphasis was on understanding the levels or prevalence of selected out- 
comes at two points in time, 1998 and 2003. The 1998 data paint a picture of the circumstances 
of residents and the conditions in Jobs-Plus developments at the start of the program. The 2003 
data describe the same developments (but not necessarily the same residents) five years later. 
The differences in these outcomes indicate how developments changed on important measures 
of community life while Jobs-Plus was under way. 

The second stage of the analysis focuses on the relative or differential change between 
the Jobs-Plus developments and the comparison developments; that is, “How do the changes in 
the Jobs-Plus developments compare with the changes in the comparison developments?” The 
net differences in these changes were computed. 

While this is a straightforward way to assess change over time, its main limitation is 
that the change observed can occur for reasons that are unrelated to Jobs-Plus. For example, a 
recent influx of new residents who have better jobs before moving into the Jobs-Plus develop- 
ments could cause residents at follow-up to look very different from their baseline counterparts, 
even in the absence of a Jobs-Plus effect. 

Indicators of Change 

Table 6.2 lists the survey-based measures used to represent six domains of quality of 
life in public housing: economic and material well-being, social capital, personal safety and vic- 
timization, neighborhood conditions, residential satisfaction, and child well-being. These con- 
structs are measured by a mix of single -item and multi-item scales. 

Economic and Material Well-Being 

Three measures are used to represent this domain: employment, household income, and 
an index of material hardship. Together, these measures reflect the economic well-being of public 
housing residents. A program like Jobs-Plus is expected to increase residents' employment and 
earnings and, thereby, to increase the income of their households. This, in turn, should reduce the 
material hardship that is experienced. Thus, to the degree that Jobs-Plus generated positive effects 
on work in some sites (St. Paul, for example), one might expect residents of the Jobs-Plus devel- 
opment to report fewer material hardships than those living in comparison developments. 
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The Jobs-Plus Demonstration 



Table 6.2 

Survey Measures for the Quality-of-Life Change Analysis 

Domain Measure 

Economic and material well-being Percentage of respondents saying they were employed at the time of 

the survey interview 

Percentage of households with annual income greater than $10,000 

Average material hardship score, based on a six-item hardship index 
with values ranging from 0 to 6. The items ask respondents if they had 
experienced any of the following hardships because their household 
was running low on money: (1) Someone needed to see a doctor, a 
dentist, or go to the hospital, but didn’t go; (2) You borrowed money 
from family or friends to make ends meet; (3) You decided not to buy 
something you really needed; (4) You couldn't pay the full amount of 
the rent; (5) You were evicted or locked out of your home because 
you could not pay the rent; and (6) You did not have enough food to 
eat. Each hardship is weighted equally. Higher values on the index 
reflect more reports of material hardship. 

Social capital Social cohesion score, based on a five-item scale including the fol- 

lowing items: (1) People in the development can be trusted; (2) Peo- 
ple in the development are willing to help their neighbors; (3) People 
in the development generally don't get along with each other; (4) Peo- 
ple in the development do not share the same values; and (5) People 
in the development live in a close-knit community. Higher values on 
the scale reflect greater social cohesion. 

Percentage of respondents reporting access to helping networks in the 
development 

Percentage of respondents reporting ties to working adults in the de- 
velopment 

Personal safety and victimization in Percentage of respondents saying they feel safe around their public 
housing development housing unit, during the day and after dark 

Percentage of respondents saying they were subject to violence, rob- 
bery, burglary, or threatened with a weapon in the past 12 months 

(continued) 



133 




Table 6.2 (continued) 



Domain 


Measure 


Problem conditions in housing 


Average problem conditions score, based on a five-item index, with 


development 


values ranging from 0 to 15. The items include: drinking in public, 
drug trafficking, gangs causing trouble, guns and gunfire, and outsid- 
ers causing trouble. Higher values on the scale mean that more re- 
spondents perceived these as big problem conditions in the develop- 
ment. 


Satisfaction with housing development 


Percentage of respondents rating the development as an excellent, 
very good, or good place to live 


Outcomes for children ages 6 to 1 1 


Percentage of children participating in school-sponsored or outside- 
school clubs and activities in the last 12 months 

Percentage of children receiving poor grades in the last 12 months 


Outcomes for children ages 12 to 17 


Percentage of children receiving poor grades in the last 12 months 
Percentage of children ever expelled or suspended from school 
Percentage of children ever in trouble with the police 



Social Capital 

Researchers have stressed the importance of social capital in creating access to eco- 
nomic opportunity. “Social capital” commonly refers to such factors as residents’ social net- 
works, their social isolation or cohesion, and their access to community supports for work. So- 
cial networks have been documented to influence individuals’ values, aspirations, and prefer- 
ences and to shape their behavior with respect to work, crime, and education. 5 The designers of 
Jobs-Plus expected the program to be easier to implement in sites where social capital was 
greater, because employment-related information, encouragement, and supports could flow (at 
least partly) through existing networks. 

To measure social capital in public housing, the Jobs-Plus surveys included questions 
about two forms of mutual exchange and connectedness: (1) those that help residents get by on 
a daily basis — for example, by providing rides, helping with groceries, or providing informal 
child care; and (2) those that help residents get ahead economically — through job leads, refer- 

5 Galster and Killen, 1995; Briggs, 1998. 
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ences, and connections to job openings. 6 In neighborhoods with concentrated poverty, both 
types of exchanges may be limited. In addition, high crime rates can keep people behind their 
doors, further limiting social exchange. 7 

Social cohesion was measured by survey questions that asked respondents whether their 
neighbors were willing to help each other, their neighborhood was close-knit, the people in the 
neighborhood could be trusted, and residents shared common values. 8 Neighborhoods that have 
a higher degree of social cohesion can better maintain informal social control to regulate the 
behaviors of residents and thereby achieve public order. 9 

Personal Safety and Victimization 

In many cities, public housing projects are located in communities with high rates of 
crime and abandonment. These unsafe environments instill fear among residents, and the fear 
experienced by residents is a key reason that they want to move out of public housing. The 
Moving To Opportunity study, which offers important insights on this front, suggests that living 
in high-crime neighborhoods takes its toll both emotionally and economically on public housing 
families, particularly mothers, who adopt safety strategies that keep them from making invest- 
ments in their own education or training. Further, residents’ deep distrust of neighbors and their 
tendency toward “keeping to themselves” prevent them from seeking informal child care ar- 
rangements in their housing developments. 10 

Two measures are used in the analysis to assess whether the Jobs-Plus and comparison 
developments became safer places to live and raise families: (1) the percentage of respondents 
saying they feel safe around their housing unit (during the day and after dark) and (2) the per- 
centage of respondents saying they were subject to violence, robbery, or burglary or were 
threatened with a weapon in the past 12 months. These two measures make it possible to con- 
trast residents’ feelings of vulnerability in their housing development with their reported levels 
of victimization, which past research suggests is quite high. 11 

Problem Conditions in the Development 

Environmental disorder (as indicated by graffiti and gangs, for example) can result in 
feelings by residents that their neighborhood is unsafe. Such disorder might signal that no one 



6 Briggs, 1998. 

7 Kling, Liebman, and Katz, 2001. 

8 Sampson, Raudenbush, and Earls, 1997. 

9 Sampson, Raudenbush, and Earls, 1997. 

10 Kling, Liebman, and Katz, 2001. 

1 'Based on a national public housing survey, Zelon (1994) estimates an annual victimization rate of 27.6 
percent for public housing households living in family developments. 
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cares about a neighborhood, making it seem an undesirable place to live or pass through. Re- 
search suggests that these negative signals can have profound effects on residents’ sense of se- 
curity, competence, and locus of control. 12 

To measure this construct, survey respondents were asked whether they considered the 
following to be “pretty big” or “very big” problems in their housing development: drinking in 
public, drug trafficking, gangs causing trouble, guns and gunfire, and outsiders causing trouble. 
A problem condition index was created to measure the extent to which residents perceived these 
problems. Values on the index ranged from 0 to 15, with higher values indicating that residents 
perceived these as serious problems in their respective developments. 

Residential Satisfaction 

Survey respondents were asked to rate the desirability of their development as a place to 
live, on a five-point scale: excellent, very good, good, not so good, and awful. The first three 
categories were combined to represent a positive rating. 

Child Outcomes 

Recent experimental evaluations of employment-focused interventions have shown that 
programs that increase parents' earnings and income can also produce positive effects for their 
children (at least for younger children between the ages of 6 and ll). 13 Thus, a program like 
Jobs-Plus may indirectly affect children by increasing parents’ income through work. For ex- 
ample, children of working parents might benefit from the increased regularity of their families’ 
routines, the enhanced self-esteem of their parent(s), or the additional income from their par- 
ents’ jobs. On the other hand, children might be negatively affected if their parents’ jobs reduce 
the amount of time available for care and supervision or unduly increase parental stress. 14 Fur- 
thermore, in the context of a place-based program like Jobs-Plus, it is possible that children 
might be positively influenced by changes in behaviors of other adults in their neighborhood. 

Measures of child well-being for the present analysis cover two broad domains: school 
performance and personal behavior. For younger children (ages 6 to 11), the measures focus on 
positive and negative school-related outcomes. For older children (ages 12 to 17), the measures 
focus on school-related outcomes plus behavioral outcomes (such as police involvement). The 
latter were not examined for younger children because of their low incidence for this group. 



l2 Mijanovich and Weitzman, 2003. 

l3 Morris and Michalopoulos, 2000; Morris, Forthcoming, 2005. 
14 Monis and Jones, 2002. 



136 





Findings on Quality-of-Life Changes 

Using the measures described above, this section examines how the quality of life of 
residents and their families in the Jobs-Plus developments and in the comparison developments 
changed over time — and how these changes compare across these developments. 

Table 6.3 characterizes the nature of change experienced in the Jobs-Plus developments 
in Baltimore, Dayton, and St. Paul using three broad categories: improved, worsened, and un- 
changed. Given the small sample sizes in this analysis, changes that are statistically significant 
are typically found to be large enough in magnitude to be substantively important. Table 6.4 
presents the detailed site-by-site estimates of change that support the characterizations. 15 The 
three categories of change are defined as follows: 

• Improved. This rating is for outcomes that improved appreciably between 
1998 and 2003. To receive the rating, the improvement had to be nontrivial 
and large enough to achieve statistical significance. 16 For example, Table 6.4 
indicates that, in 1998, 60.4 percent of the respondents from the St. Paul Jobs- 
Plus development were living in households with annual incomes of $10,000 
or more. By 2003, 73.8 percent of the households from that site were in this 
category. The difference of 13 percentage points is statistically significant. 

• Worsened. This rating is used to indicate that an outcome worsened appre- 
ciably between 1998 and 2003 and that the magnitude of change was large 
enough to achieve statistical significance. Consider how survey respondents 
in the Baltimore Jobs-Plus development rated their satisfaction with their 
housing development. Table 6.4 indicates that 58.9 percent of the baseline 
survey respondents rated their development favorably, whereas only 41.1 
percent of the follow-up survey respondents did so. The 17.8 percentage 
point difference represents an appreciable decline in perceived desirability. 



15 Table 6.4 presents two distinct pieces of information for the three Jobs-Plus developments: ( 1) 1998 and 
2003 outcome levels and (2) estimates of change over time. The levels tell us what circumstances were like for 
residents and the developments at the time each survey was fielded. The change over time calculations signal 
whether conditions or circumstances were getting better or getting worse for these people and places over the 
course of the demonstration. As a place-based intervention, Jobs-Plus is designed to improve the quality of life 
of residents in public housing. Since “change” can be either positive or negative, it is important to keep the 
outcome of interest in mind when evaluating how things changed. For example, on one hand, a decline in ma- 
terial hardship would be considered a change in the positive direction, but a decline in household income, on 
the other hand, would not be good news for this outcome. 

16 The criterion of statistical significance is used for convenience, to deal with the issue of using different 
metrics for the different measures in the analysis. 
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The Jobs-Plus Demonstration 
Table 6.3 

Summary of Quality-of-Life Changes 
in the Jobs-Plus Developments from 1998 to 2003 



Jobs-Plus Developments 


Measure 


Baltimore 


Dayton 


St. Paul 


Did conditions improve, worsen, or remain the same, according to residents ' reports 


9 




Economic and material well-beins 

Currently employed (%) 


Unchanged 


Unchanged 


Improved 


Households with annual income of $10,000 
or more (%) 


Improved 


Unchanged 


Improved 


Average material hardship score 


Worsened 


Worsened 


Unchanged 


Social capital 

Average score on Social Cohesion Index 


Unchanged 


Unchanged 


NA 


Respondent can rely on someone in the 
development for help (%) 


Unchanged 


Unchanged 


NA 


Some or all of the adults in the development 
well-known to the respondent have 
full-time, steady paid work (%) 


Improved 


Unchanged 


NA 


Safety and victimization 

Feel "very” or "somewhat" safe being alone, 
near the unit, during the day or after dark (%) 


Unchanged 


Unchanged 


NA 


Victim of violence, robbery, burglary, or 
threatened with a weapon in the past 12 months (%) 


Unchanged 


Unchanged 


Unchanged 


Conditions within development 

Average problem conditions score 


Worsened 


Worsened 


NA 


Residential satisfaction 

Development is an "excellent,” "very good," 
or "good” place to live (%) 


Worsened 


Worsened 


Unchanged 


Outcomes for children ages 6 to 11 

Participated in school-sponsored or outside- 
school activities (%) 


Unchanged 


Improved 


NA 


Received poor grades (%) 


Improved 


Unchanged 


NA 


Outcomes for children ages 12 to 17 

Received poor grades (%) 


Unchanged 


Improved 


NA 


Ever suspended or expelled from school (%) 


Unchanged 


Unchanged 


NA 


Ever got into trouble with police (%) 


Unchanged 


Unchanged 


NA 



SOURCES: MDRC summary based on the Jobs-Plus baseline (1998) and follow-up (2003) surveys. 
NOTE: See Table 6.2 for information on measures reported in this table. 
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The Jobs-Plus Demonstration 
Table 6.4 

Estimates of Quality-of-Life Changes in the Jobs-Plus Developments from 1998 to 2003 







Baltimore 




Dayton 






St. Paul 3 




Measure 


1998 


2003 


Change 


1998 


2003 


Change 


1998 


2003 


Change 


Economic and material well-beine 

Currently employed (%) 


53.2 


57.5 


4.3 


61.0 


61.3 


0.3 


53.6 


63.7 


10.1 * 


Households with annual income of $10,000 or more (%) 


21.7 


31.4 


9.8 ** 


25.4 


28.7 


3.3 


60.4 


73.8 


13.4 ** 


Average material hardship score (ranges from 0 to 6) 


2.4 


2.8 


0.4 ** 


2.2 


2.5 


0.3 * 


1.5 


1.4 


-0.1 


Social capital 

Average score on the Social Cohesion Index 
(ranges from 0 to 15) 


7.2 


6.9 


-0.3 


6.6 


6.4 


-0.2 


NA 


NA 


NA 


Respondent can rely on someone in the development 
for help (%) 


82.0 


78.0 


-4.0 


76.7 


74.4 


-2.3 


NA 


NA 


NA 


Some or all of the adults in the development well-known 
to the respondent have full-time, steady paid work (%) 


38.5 


50.0 


11.5 ** 


45.2 


47.3 


2.1 


NA 


NA 


NA 


Safety and victimization 

Feel "very" or "somewhat" safe being alone, 
near the unit, during the day or after dark (%) 


36.6 


35.6 


-1.0 


46.6 


47.5 


0.9 


NA 


NA 


NA 


Victim of violence, robbery, burglary, or threatened with 
a weapon in the past 12 months (%) 


29.0 


29.2 


0.2 


37.4 


40.3 


2.9 


23.8 


21.2 


-2.6 


Conditions within development 

Average problem conditions score (ranges from 0 to 15) 


8.9 


10.3 


j 4 *** 


8.7 


10.1 


1 4 *** 


NA 


NA 


NA 


Residential satisfaction 

Development is an "excellent," "very good," or "good" 
place to live (%) 


58.9 


41.1 


-17 8 *** 


60.4 


44.8 


-15 7 *** 


88.7 


87.6 


-1.1 



(continued) 






Table 6.4 (continued) 







Baltimore 




Dayton 






St. Paul a 




Measure 


1998 


2003 


Change 


1998 


2003 


Change 


1998 


2003 


Change 


Outcomes for children a ties 6 to ll b 




















Participated in school-sponsored or outside- 
school activities (%) 


64.2 


61.6 


-2.6 


52.3 


66.3 


13.9 * 


NA 


NA 


NA 


Received poor grades (%) 


22.9 


11.6 


-11.3 * 


23.4 


21.6 


-1.8 


NA 


NA 


NA 


Outcomes for children a ties 12 to 17 c 

Received poor grades (%) 


31.3 


28.6 


-2.7 


41.7 


21.4 


-20.2 ** 


NA 


NA 


NA 


Ever suspended or expelled 
from school (%) 


44.8 


34.6 


-10.2 


31.7 


33.3 


1.7 


NA 


NA 


NA 


Ever got into trouble with police (%) 


11.9 


9.6 


-2.3 


11.7 


14.0 


2.4 


NA 


NA 


NA 


Sample size 


219 


219 




241 


181 




151 


113 





SOURCES: MDRC calculations using data from the Jobs-Plus baseline (1998) and follow-up (2003) surveys. 



NOTES: See Table 6.2 for information on measures reported in this table. 

Statistical significance levels are indicated as *** = 1 percent; ** = 5 percent; and * = 10 percent. 

Actual sample sizes for individual measures may vary as a result of missing data. 

'‘Many of the questions included on the core survey instrument were omitted from the St. Paul survey because they could not be translated easily into 
Hmong, the language spoken by a large majority of residents living in the Jobs-Plus development. The missing items are denoted by "NA” in the table. 

b The sample sizes for the 1998 child outcome measures for children ages 6 to 1 1 in Baltimore are 110 in the Jobs-Plus group and 107 in the 
comparison group. The 2003 sample sizes for Baltimore are 87 in the Jobs-Plus group and 125 in the comparison group. For Dayton, the sample sizes for 
the 1998 child outcomes measures for children in the 6 to 1 1 age range are 129 in the Jobs-Plus group and 206 in the comparison group. The 2003 sample 
sizes are 91 in the Jobs-Plus group and 144 in the comparison group. 

c The sample sizes for child outcome measures for children ages 12 to 17 vary by site: 67 in the Jobs-Plus group and 54 in the comparison group in 
Baltimore in 1998; 52 in the Jobs-Plus group and 1 10 in the comparison group in Baltimore in 2003; 60 in the Jobs-Plus group and 84 in the comparison 
group in Dayton in 1998; and 57 in the Jobs-Plus group and 1 1 1 in the comparison group in Dayton in 2003. 






• Unchanged. This classification implies that there was very little difference 
in how a condition was rated by respondents to the baseline and the follow- 
up surveys. 

Overall Finding: Mainly Small or No Changes in Quality of Life 

What conclusions can one draw about how the Jobs-Plus developments fared over time 
with respect to quality of life? Before turning to the details, it is worth noting that the most strik- 
ing finding emerging from Tables 6.3 and 6.4 is one of little overall change. In Baltimore and 
Dayton, there is almost no evidence that “things got better.” In St. Paul, however, where there is 
clear evidence that Jobs-Plus increased earnings at both the individual and the development lev- 
el (see Chapters 4 and 5), there is some evidence of quality-of-life improvements. 

St. Paul: Some Small Improvements in Material Quality of Life 

On economic outcomes, which are most directly affected by Jobs-Plus, the St. Paul 
Jobs-Plus development registered moderate improvements between the first and second 
waves of the survey. For example, Table 6.4 indicates a 10 percentage point increase in the 
proportion of respondents who were employed at the time of the follow-up survey. 17 In addi- 
tion, the table indicates a 13 percentage point increase in the proportion of households with 
annual incomes over $10,000. 18 

However, these gains in employment and income translated into mixed improvements 
in material well-being, based on survey respondents reports of six types of hardships within the 
past months: 19 (1) needing to see a doctor or a dentist or to go to the hospital but being unable to 
afford it, (2) borrowing money from family or friends to make ends meet, (3) deciding not to 
buy something that was really needed, (4) not being able to pay the full monthly rent, (5) being 
evicted or locked out because of rent arrears, and (6) not having enough food to eat. 20 Computa- 
tions not reported in the table indicate that 75 percent of the 1998 survey respondents from St. 
Paul reported one or more of these hardships, whereas only 66 percent of the 2003 survey re- 
spondents did so — a decline of 9 points in the percentage of respondents who reported at least 
one hardship. However, the average number of hardships experienced (the material hardship 
index in Table 6.4) changed very little — from 1.5 reported hardships per respondent, on aver- 
age, in 1998 to 1.4 in 2003. 

n These levels are based on respondents’ self-reported employment status. 

ls This benchmark was not inflation-adjusted across the two surveys. 

19 All respondents in the follow-up survey and most in the baseline survey would have been living in pub- 
lic housing during the reference period. 

20 Survey respondents were asked whether they had experienced each of these hardships during the 12 
months before their interview. 
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To explore this issue further, Figure 6.1 displays the percentage of survey respondents 
who reported three specific material hardships in 1998 and 2003. For two of the hardships (rent 
arrears and reliance on others for cash), the Jobs-Plus development in St. Paul experienced a 4 
to 5 percentage point increase in incidence. But for the third hardship (inadequate food), there 
was a 16 percentage point decline. This suggests that as households’ incomes increased in St. 
Paul, families were better able to meet their food-related needs but that they did not attain a de- 
gree of economic self-sufficiency to overcome some of the other hardships. 

Many of the questions asked on the core survey were omitted from the St. Paul survey 
instrument because they could not be translated easily into Hmong, the predominant language 
spoken by residents living in this development. As a result, it was not always possible to get at 
all the measures of community life that are reported for the other sites. However, it was possible 
to ask residents whether they had been subjected to a crime in the past 12 months and how they 
rated their housing development as a place to live. The findings presented in Table 6.4 indicate 
that victimization rates declined very slightly in this site, from 24 percent to 21 percent. Further, 
based on accounts of survey respondents (see Figure 6.2), it appears that this development ex- 
perienced small to modest improvements in the presence of conditions that heighten risk for 
individuals and families. For example, reports of gangs causing trouble in the development de- 
clined by 10 percentage points, from 33 percent in 1998 to 23 percent in 2003, a change statisti- 
cally significant at the .05 level of significance. Reports of people selling drugs in public also 
declined over time, from 15 percent in 1998 to 9 percent in 2003, but this change was not statis- 
tically significant. Overall, dangerous conditions were far less prevalent in St. Paul Jobs-Plus 
than in the other two sites and this pattern held true over time. 21 

This picture of the Jobs-Plus development in St. Paul — painted by residents living 
there in 1998 and 2003 — does not lead one to conclude that the changes experienced were par- 
ticularly dramatic or transformative. On some measures, such as those that tap aspects of eco- 
nomic well-being or community context, there was some improvement. But on other, broader 
indicators of residential satisfaction and community life, there was very little change. 

Baltimore and Dayton: Largely Unchanging or Worsening Circumstances 

Jobs-Plus in Baltimore had no development-level impacts, and Jobs-Plus in Dayton had 
moderate development-level impacts. Thus, one would not expect major spillover effects on 
other community outcomes in these sites. Survey findings appear to confirm this expectation. 



“'There are slight differences in how this question is asked in the three study sites. The Baltimore and 
Dayton surveys asked respondents to rate how “big” of a problem these various conditions were in their re- 
spective developments. Due to translation constraints, the St. Paul survey just asks respondents to indicate 
whether they perceived these conditions as problems in their respective developments. 
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The Jobs-Plus Demonstration 



Figure 6.1 

Changes Over Time on Selected Measures of Material Hardship for Jobs-Plus Residents 



The percentage of survey respondents reporting “some” or “a lot” of the following hardships in the past 12 months 

due to financial constraints 



Borrowed cash 
from family or 
friends 



Baltimore 

100 




Dayton 

100 




St, Paul 

100 




Could not pay the 
full rent 




Did not have 
enough food 
to eat 



100 

75 

50 

25 

0 l 



100 



100 



31 



43 







2003 



SOURCES: MDRC calculations using data front the Jobs-Plus baseline (1998) and follow-up (2003) surveys. 




The Jobs-Plus Demonstration 



Figure 6.2 



Changes Over Time in Problem Conditions in the Jobs-Plus Developments 

The percentage of survey respondents rating the following conditions as “pretty big” or “very big” problems in the development 
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SOURCES: MDRC calculations using data from the Jobs-Plus baseline (1998) and follow-up (2003) surveys. 




Tables 6.3 and 6.4 indicate that, on most of the 15 measures reported, the Jobs-Plus de- 
velopments in both sites showed little or no improvement between 1998 and 2003. For exam- 
ple, residential satisfaction declined; problem conditions in the developments increased; and 
material hardship went up (see also Figures 6.1 and 6.2). In addition, there was no notable im- 
provement in the general social climate, victimization rates, or perceptions of personal safety. 

With the exception of a few measures, most of the child outcomes remained unchanged. 
Children in Dayton and Baltimore — the two sites where it was possible to collect survey data 
on child well-being — registered modest improvements over time in school performance: There 
was an 1 1 percentage point decline in the proportion of children ages 6 to 1 1 in Baltimore who 
received poor grades, and a 20 percentage point decline in this outcome was observed for older 
children (ages 12 to 17) in Dayton. Younger children in Dayton were also more likely to par- 
ticipate in school-sponsored or outside-school activities. School suspension rates dropped by 10 
percentage points in Baltimore, but the absolute levels of this outcome remained relatively high 
in both sites. While these generally positive outcomes for younger and older children are en- 
couraging, it is unclear whether these improvements can be linked to Jobs-Plus, particularly in a 
site like Baltimore, where the program had no effect on parents’ work outcomes. 

Quality-of-Life Changes in the Jobs-Plus Developments Mirror the 
Changes in the Comparison Developments 

Table 6.5 summarizes how the quality-of-life changes for the Jobs-Plus developments 
between 1998 and 2003 compare with the same changes for their comparison developments. 
For each outcome and site, the differences in change over time are classified into one of three 
categories: same, better, or worse. 22 This table tells a story of strikingly similar changes for the 
Jobs-Plus and comparison developments. Regardless of the outcome measure, the Jobs-Plus 
developments and the comparison developments almost always changed by about the same 
magnitude and in the same direction. 

Table 6.6 presents estimates of differences in change between the Jobs-Plus and the 
comparison developments. Three sets of estimates are presented for each outcome measure: (1) 
the 1998 and 2003 levels for the Jobs-Plus developments, (2) the 1998 and 2003 levels for the 



22 Again, note that the criterion of statistical significance is used for convenience, to deal with the issue of 
using different metrics for the different measures in the analysis. In all cases where the changes are statistically 
significant, they are typically found to be large enough in magnitude to be substantively important. 
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The Jobs-Plus Demonstration 
Table 6.5 



Summary of Differences in Changes Over Time in Quality-of-Life Measures: 
Jobs-Plus Versus Comparison Developments, 1998 to 2003 



Measure 


Baltimore 


Dayton 


St. Paul 


How did the Jobs-Plus developments change relative to the change in comparison developments? 
Did they fare the same, better, or worse? 




Economic and material well-beins 

Currently employed (%) 


Same 


Same 


Same 


Households with annual income of $10,000 
or more (%) 


Same 


Same 


Same 


Average material hardship score 


Same 


Same 


Same 


Social capital 

Average score on the Social Cohesion Index 


Same 


Same 


NA 


Respondent can rely on someone in the 
development for help (%) 


Same 


Same 


NA 


Some or all of the adults in the development 
well-known to the respondent have 
full-time, steady paid work (%) 


Better 


Better 


NA 


Safety and victimization 

Feel "very" or "somewhat" safe being alone, 
near the unit, during the day or after dark (%) 


Same 


Better 


NA 


Victim of violence, robbery, burglary, or 
threatened with a weapon in the past 12 months (%) 


Same 


Same 


Same 


Conditions within development 








Average problem conditions score 


Same 


Same 


NA 


Residential satisfaction 

Development is an "excellent," "very good," 
or "good" place to live (%) 


Same 


Same 


Worse 


Outcomes for children ages 6 to 11 

Participated in school-sponsored or outside- 
school activities (%) 


Same 


Same 


NA 


Received poor grades (%) 


Better 


Same 


NA 


Outcomes for children ages 12 to 17 

Received poor grades (%) 


Same 


Same 


NA 


Ever suspended or expelled from school (%) 


Same 


Same 


NA 


Ever got into trouble with police (%) 


Same 


Same 


NA 



SOURCES: MDRC calculations using data from the Jobs-Plus baseline (1998) and follow-up (2003) surveys. 



NOTE: See Table 6.2 for information on measures reported in this table. 
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The Jobs-Plus Demonstration 
Table 6.6 



Estimates of Differences in Changes Over Time in Quality-of-Life Measures: 
Jobs-Plus Versus Comparison Developments, 1998 to 2003 



Measure 






Baltimore 




Dayton 




St. Paul 8 


Development 


1998 


2003 


Change 


1998 


2003 


Change 


1998 


2003 


Change 


Economic and material well-being 






















Currently employed (%) 


Jobs-Plus 


53.2 


57.5 


4.3 


61.0 


61.3 


0.3 


53.6 


63.7 


10.1 * 




Comparison 


51.0 


52.2 


1.2 


61.8 


56.9 


-4.8 


52.1 


63.7 


11.6 * 




Net Difference 






3.1 






5.2 






-1.6 


Households with annual income of $10,000 or more (%) 


Jobs-Plus 


21.7 


31.4 


9.8 ** 


25.4 


28.7 


3.3 


60.4 


73.8 


13.4 ** 




Comparison 


21.6 


33.0 


\ \ 4 *** 


28.3 


23.9 


-4.4 


45.1 


60.9 


15.8 ** 




Net Difference 






-1.6 






7.7 






-2.4 


Average material hardship score (ranges from 0 to 6) 


Jobs-Plus 


2.4 


2.8 


0.4 ** 


2.2 


2.5 


0.3 * 


1.5 


1.4 


-0.1 




Comparison 


1.8 


2.5 


Q J *** 


2.2 


2.6 


Q 4 *** 


1.7 


L6 


-0.1 




Net Difference 






- 0.3 






-0.1 






0.0 


Social capital 






















Average score on the Social Cohesion 






















Index (ranges from 0 to 15) 


Jobs-Plus 


7.2 


6.9 


-0.3 


6.6 


6.4 


-0.2 


NA 


NA 


NA 




Comparison 


7.8 


7.2 


-0.6 ** 


6.3 


6.3 


0.0 


NA 


NA 


NA 




Net Difference 






0.3 






-0.3 






NA 


Respondent can rely on someone in the 






















development for help (%) 


Jobs-Plus 


82.0 


78.0 


-4.0 


76.7 


74.4 


-2.3 


NA 


NA 


NA 




Comparison 


85.4 


78.1 


_7 2 ** 


77.5 


72.7 


-4.7 


NA 


NA 


NA 




Net Difference 






3.2 






2.5 






NA 


Some or all of the adults in the development well-known 






















to the respondent have full-time, steady paid work (%) 


Jobs-Plus 


38.5 


50.0 


11.5 ** 


45.2 


47.3 


2.1 


NA 


NA 


NA 




Comparison 


52.2 


50.7 


-1.5 


41.9 


32.8 


-9.1 ** 


NA 


NA 


NA 




Net Difference 






13.1 ** 






11.2 * 






NA 
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Table 6.6 (continued) 



Measure 






Baltimore 




Dayton 




St. Paul 2 


Development 


1998 


2003 


Change 


1998 


2003 


Change 


1998 


2003 


Change 


Safety and victimization 






















Feel "very" or "somewhat" safe being alone. 






















near the unit, during the day or after dark (%) 


Jobs-Plus 


36.6 


35.6 


- 1.0 


46.6 


47.5 


0.9 


NA 


NA 


NA 




Comparison 


54.4 


50.3 


-4.1 


56.0 


41.8 


_14 2 *** 


NA 


NA 


NA 




Net Difference 






3.1 






15.1 ** 






NA 


Victim of violence, robbery, burglary, or 






















threatened with a weapon (%) 


Jobs-Plus 


29.0 


29.2 


0.2 


37.4 


40.3 


2.9 


23.8 


21.2 


- 2.6 




Comparison 


22.1 


25.1 


3.0 


39.1 


42.2 


3.1 


21.0 


22.0 


1.0 




Net Difference 






-2.8 






-0.2 






-3.6 


Average problem conditions score (ranges from 0 to 15) 


Jobs-Plus 


8.9 


10.3 


1 4 *** 


8.7 


10.1 


1 4 *** 


NA 


NA 


NA 




Comparison 


6.6 


8.8 


2 i *** 


8.4 


9.5 


1 1 *** 


NA 


NA 


NA 




Net Difference 






-0.8 






0.3 






NA 


Residential satisfaction 






















Housing development is an "excellent," "very good," 






















or "good" place to live (%) 


Jobs-Plus 


58.9 


41.1 


-17 8 *** 


60.4 


44.8 


-15 7 *** 


88.7 


87.6 


- 1.1 




Comparison 


58.9 


41.1 


-17 8 *** 


55.9 


34.2 


-21 8 *** 


80.5 


94.5 


14 0 *** 




Net Difference 






1.6 






6.1 






-15.1 ** 


Outcomes for children ases 6 to 11 b 






















Participated in school-sponsored or outside- 


Jobs-Plus 


64.2 


61.6 


- 2.6 


52.3 


66.3 


13.9 * 


NA 


NA 


NA 


school activities (%) 


Comparison 


85.8 


71.2 


-14.6 ** 


52.7 


57.4 


4.7 


NA 


NA 


NA 




Net Difference 






12.1 






9.2 






NA 


Received poor grades (%) 


Jobs-Plus 


22.9 


11.6 


-11.3 * 


23.4 


21.6 


- 1.8 


NA 


NA 


NA 




Comparison 


17.3 


21.1 


3.8 


26.4 


19.0 


-7.4 


NA 


NA 


NA 




Net Difference 






-15.1 * 






5.5 






NA 
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Table 6.6 (continued) 



Measure 






Baltimore 




Dayton 




St. Paul" 


Development 


1998 


2003 


Change 


1998 


2003 


Change 


1998 


2003 


Change 


Outcomes for children ages 12 to 17 c 






















Received poor grades (%) 


Jobs-Plus 


31.3 


28.6 


-2.7 


41.7 


21.4 


-20.2 ** 


NA 


NA 


NA 




Comparison 


29.2 


24.3 


-4.9 


37.3 


32.7 


-4.7 


NA 


NA 


NA 




Net Difference 






2.2 






-15.6 






NA 


Ever suspended or expelled 


Jobs-Plus 


44.8 


34.6 


-10.2 


31.7 


33.3 


1.7 


NA 


NA 


NA 


from school (%) 


Comparison 


31.4 


30.9 


-0.5 


14.7 


16.5 


1.8 


NA 


NA 


NA 




Net Difference 






-9.7 






-0.8 






NA 


Ever got into trouble with police (%) 


Jobs-Plus 


11.9 


9.6 


-2.3 


11.7 


14.0 


2.4 


NA 


NA 


NA 




Comparison 


3.9 


7.3 


3.4 


0.8 


0.0 


-0.8 


NA 


NA 


NA 




Net Difference 






-5.7 






-4.4 






NA 


Sample size 


Jobs-Plus 


219 


219 




241 


181 




151 


113 






Comparison 


215 


323 




287 


275 




119 


91 





SOURCES: MDRC calculations using data from the Jobs-Plus baseline (1998) and follow-up (2003) surveys. 



NOTES: See Table 6.2 for information on measures reported in this table. 

Statistical significance levels are indicated as *** = 1 percent; ** = 5 percent; and * = 10 percent. 

Actual sample sizes for individual measures may vary as a result of missing data. 

a Many of the questions included on the core survey instrument were omitted from the St. Paul survey because they could not be translated easily into 
Hmong, the language spoken by a large majority of residents living in the Jobs-Plus development. The missing items are denoted by “NA” in the table. 

b The sample sizes for the 1998 child outcome measures for children ages 6 to 1 1 in Baltimore are 110 in the Jobs-Plus group and 107 in the comparison 
group. The 2003 sample sizes for Baltimore are 87 in the Jobs-Plus group and 125 in the comparison group. For Dayton, the sample sizes for the 1998 child 
outcomes measures for children in the 6 to 1 1 age range are 129 in the Jobs-Plus group and 206 in the comparison group. The 2003 sample sizes are 91 in the 
Jobs-Plus group and 144 in the comparison group. 

c The sample sizes for child outcome measures for children ages 12 to 17 vary by site: 67 in the Jobs-Plus group and 54 in the comparison group in 
Baltimore in 1998; 52 in the Jobs-Plus group and 110 in the comparison group in Baltimore in 2003; 60 in the Jobs-Plus group and 84 in the comparison 
group in Dayton in 1998; and 57 in the Jobs-Plus group and 1 1 1 in the comparison group in Dayton in 2003. 







comparison developments, and (3) the net difference in change between the Jobs-Plus and com- 
parison developments. 23 The net difference in change is calculated by comparing the mean 
change (1998 to 2003) in a measure for the Jobs-Plus development to the mean change for the 
comparison developments. 

These findings show little support for the spillover hypothesis, because both the Jobs- 
Plus developments and the comparison developments experienced very similar change trajecto- 
ries. In most cases, if the Jobs-Plus developments experienced small changes over time, so did 
the comparison developments. And where the Jobs-Plus developments experienced reasonably 
large changes, the comparison developments did, too. Even in St. Paul — the one case where it 
seemed most plausible to expect a link between program effects on earnings and community 
spillovers effects — there is little evidence of transformative changes caused by the program. 



Summary and Conclusions 

As a place-based employment intervention, Jobs-Plus was motivated by the vision that 
dramatic increases in employment and earnings among public housing residents would improve 
their quality of life and create spillovers that would result in improvements in important aspects of 
the quality of life in the development. This chapter examines the extent to which such spillover 
effects materialized in important aspects of community life in three sites that participated in this 
demonstration. In two of the three sites examined (Baltimore and Dayton), Jobs-Plus did not pro- 
duce large development-level earnings gains — the fundamental precondition for spillover effects 
— and so these sites are probably not good cases for assessing the hypothesis of community spill- 
over, thus leaving St. Paul as the best of the three sites for testing this hypothesis. However, even 
there, the development-level earnings effects of Jobs-Plus were not extraordinary. 



23 In Table 6.6, the net difference in change is shown in bold type to make it stand out from the other esti- 
mates repotted in the table. 
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Chapter 7 

The Lessons and Implications of Jobs-Plus 



This study demonstrates that an employment-focused intervention based in public hous- 
ing developments can work. Although challenging to implement, Jobs-Plus, when operated 
well, increased public housing residents’ earnings relative to what those earnings would have 
been without the program. This suggests that the Jobs-Plus strategy can help achieve the resi- 
dent self-sufficiency objectives of the 1998 federal housing reform law as well as help advance 
the work-promoting mission of welfare and workforce agencies whose target populations in- 
clude residents of public housing. 

This concluding chapter puts the evaluation findings into a broader context and takes a 
closer look at a number of policy questions that the findings inspire. For example, what do the 
data suggest in general about public housing residents’ labor force attachment? How do the ef- 
fects of Jobs-Plus compare with those of other programs? Can a program like Jobs-Plus be ef- 
fective for different types of public housing residents? Did Jobs-Plus contribute to the goals of 
welfare reform? Would the program have been more effective if participation in it were manda- 
tory? Did the rent incentives matter? Can Jobs-Plus help build more mixed-income communi- 
ties within public housing? Can it be replicated and effective on a broader scale? How much 
does it cost to operate Jobs-Plus, and would replicating it be a good investment? Finally, what 
lessons from Jobs-Plus apply outside public housing? 

How strongly are public housing residents attached to the labor force? 

One of the most striking findings from this study has nothing to do with the program’s 
effectiveness. Rather, it is the fact that, in all six of the Jobs-Plus sites, public housing residents 
in the program and the comparison developments saw their rates of employment and earnings 
climb sharply over the course of the 1990s, even before the launch of Jobs-Plus in 1998. These 
rising rates were most likely driven by an improving economy and federal policy reforms, such 
as welfare reform and the expansion of the Earned Income Tax Credit (EITC). The upward 
trends leveled off after the expanding economy began to contract with the onset of a national 
recession in 2001, although they generally did not fall back to their previous lower levels. 

The positive trajectories reveal residents’ substantial attachment to the formal labor 
market, even in some of the nation’s poorest housing developments and even in the absence of 
Jobs-Plus. Indeed, the Unemployment Insurance wage records show that, across the Jobs-Plus 
and comparison developments, from almost 75 percent (in Baltimore) to about 90 percent (in 
Dayton) of the 1998 cohort of 21- to 61 -year-old residents worked in the formal economy at 
least at some point after Jobs-Plus began. Thus, the majority of residents wanted to — and did 
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— work, even if their work was not always steady or in high-quality jobs. This responsiveness 
of public housing residents, who are often thought to be isolated in high-poverty communities 
from broader economic opportunities, exceeded what the program’s designers had anticipated. 

The impressive growth in residents’ quarterly employment rates before Jobs-Plus 
even began reduced the margin for the program to make further improvement. Still, there re- 
mained some room for improvement. Although Jobs-Plus appears to have raised employment 
rates, larger effects might have been possible. To achieve them, the sites would have had to 
reach the minority of residents who, despite the program’s best efforts, remained outside of 
the formal labor market. 1 

How large are Jobs-Plus’s earnings impacts relative to those of other 
employment interventions? 

The effects of Jobs-Plus on residents’ earnings were substantial, sustained, and statisti- 
cally significant. How does this accomplishment stack up against what other programs have 
achieved? With a dearth of impact studies in the housing and community development fields to 
serve as benchmarks, it is instructive to consider how Jobs-Plus’s impacts compare with those 
of employment initiatives outside public housing. Such a comparison suggests that when Jobs- 
Plus was well implemented, its earnings impacts are at the high end of those found across a 
large collection of important employment interventions that have been subjected to random as- 
signment evaluations. 

Table 7.1 presents results of 30 different interventions evaluated by MDRC using ran- 
dom assignment research designs. Most of these studies evaluated welfare-to-work and other 
welfare reform initiatives, but a few operated outside the welfare system and served a broader 
low-income population. Most of the interventions offered employment and training services; 
most included a participation mandate; and two included a time limit on cash assistance. Some 
of the interventions also offered financial incentives to help increase the net income that partici- 
pants would realize by taking a low-wage job. 2 The table orders the interventions according to 
the size of their average annual impacts on earnings measured over a three-year follow-up pe- 
riod starting from the point of random assignment. It divides the entire set of interventions into 
three tiers of 10 each, listing those with larger impacts first. To facilitate comparisons, the table 
includes in the top panel the impacts of Jobs-Plus during the first three years of the four years of 
follow-up subsequent to the program rollout period. 

'An analysis of the baseline survey of residents found that health-related problems were a substantial em- 
ployment barrier among residents who had not recently worked (Martinez, 2002). 

2 These incentives usually took the form of enhanced earnings disregards that allowed employed welfare 
recipients to keep more of their grants (which, otherwise, would have been reduced in proportion to their in- 
crease in earnings). In other cases, the incentives took the form of direct wage subsidies. 
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The Jobs-Plus Demonstration 
Table 7.1 



Summary of Impacts on Earnings of Welfare-to-Work and 
Other Employment Interventions 





Year 1 


Year 2 


Year 3 


Average for 


Percentage 


Program 


($) 


($) 


($) 


Years 1-3 ($) 


Change (%) 


Jobs-Plus 












All sites combined 


452 *** 


619 ** 


440 ** 


507 


6.3 


Stronger implementation sites 


724 ** 


1.135 *** 


2 271 *** 


1,007 


12.3 


GAIN Riverside 


1,516 *** 


2 744 *** 


1.360 *** 


1,540 


63.3 


GAIN Butte 


1,023 ** 


2 594 *** 


1,700 *** 


1,439 


64.6 


NEWWS Portland 


g24 *** 


2 477 *** 


1,634 *** 


1,312 


35.5 


SSP Plus 


1,011 *** 


1 198 *** 


1,209 *** 


1,139 


50.9 


L.A. Jobs-First GAIN 


900 *** 


1,009 *** 


NA 


954 


27.7 


SSP Plus Comparison 


658 *** 


1,182 *** 


886 ** 


908 


39.4 


MFIP 


745 *** 


992 *** 


654 * 


797 


21.7 


SSP 


502 *** 


1 077 *** 


744 *** 


774 


29.4 


GAIN San Diego 


456 ** 


922 *** 


860 *** 


743 


21.3 


Jobs First 


359 ** 


990 *** 


837 *** 


728 


12.5 


NEWWS Riverside FFA 


881 *** 


590 *** 


473 ** 


682 


23.8 


FTP 


229 


743 *** 


1 049 *** 


673 


17.6 


SWIM 


458 ** 


394 *** 


646 ** 


666 


20.1 


New Hope 


1,060 ** 


109 


798 


656 


9.6 


GAIN Alameda 


265 


650 * 


962 ** 


625 


27.1 


NEWWS Atlanta LFA 


442 *** 


578 *** 


582 ** 


534 


15.4 


NEWWS Grand Rapids LFA 


513 *** 


482 ** 


393 


463 


14.9 


NEWWS Grand Rapids HCD 


211 


597 *** 


472 * 


460 


12.9 


NEWWS Detroit 


141 


480 ** 


693 *** 


438 


12.8 


WRP 


440 ** 


173 


659 * 


424 


13.0 


NEWWS Columbus Integrated 


64 


596 *** 


499 ** 


386 


7.7 


Project Independence 


466 *** 


296 ** 


NA 


381 


13.5 


NEWWS Riverside HCD 


295 * 


249 


571 *** 


371 


17.9 


NEWWS Atlanta HCD 


85 


429 ** 


534 ** 


349 


9.0 


NEWWS Columbus Traditional 


142 


482 ** 


395 


340 


7.0 


GAIN Tulare 


-253 


126 


794 ** 


222 


6.7 


GAIN Los Angeles 


-17 


159 


189 


110 


4.9 


NEWWS Oklahoma City 


94 


152 


28 


91 


4.1 


WRP Incentives Only 


214 


6 


-79 


47 


3.1 


MFIP Incentives Only 


227 


-229 


-219 


-74 


0.0 



(continued) 
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Table 7.1 (continued) 



SOURCE: MDRC calculations using information presented in Bloom and Michalopoulos (2001). 

NOTES: Findings are reported in 2003 dollars. Impact estimates are the differences in earnings between sample 
members randomly assigned to a program or control group in each study. Results are for all sample members 
who had ever received welfare prior to random assignment. 

"Year 1" refers to the four quarters after the calendar quarter of random assignment. 

Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; and * = 10 percent. 

Outcomes indicated as NA were not measured. 

"Percentage Change" refers to the impact estimate divided by the control group mean. It is estimated here 
as the average of the percentage changes for Years 1, 2, and 3. If Year 3 earnings were not measured, the mean 
earnings and the percentage change for that program were calculated using the data from Years 1 and 2. 

The acronyms are spelled out in alphabetical order: FTP refers to the Family Transition Program; GAIN 
refers to Greater Avenues for Independence; HCD refers to Human Capital Development; LFA refers to Labor 
Force Attachment; MFIP refers to the Minnesota Family Investment Program; NEWWS refers to the National 
Evaluation of Welfare-to-Work Strategies; SSP refers to the Self-Sufficiency Project; SWIM refers to the 
Saturation Work Initiative Model; and WRP refers to the Welfare Restructuring Project. 



The table shows that the earnings impacts produced by the top tier of interventions 
ranged from $728 per year for the Connecticut Jobs First program to $1,540 per year for the 
Riverside, California, GAIN program. (All impact estimates were inflation-adjusted to 2003 
dollars.) The Jobs-Plus results averaged together for the three stronger implementation sites 
(Dayton, Los Angeles, and St. Paul) fall well within that range; in fact, at $1,007 per year across 
the first three years after the program rollout period, they fall within the upper half of that top 
tier. 3 When all six Jobs-Plus sites are included, the average annual earnings impacts rank in the 
middle of all the interventions listed in the table. 4 

As an alternative benchmark, the Jobs-Plus results were compared with the findings of 
a meta-analysis applied to 3 1 impact analyses conducted by a number of research organizations 
on 15 different voluntary employment and training programs for a wide variety of disadvan- 
taged groups (not just welfare recipients). This comparison shows, again, that the Jobs-Plus re- 



3 When the even-larger fourth-year results are included, the average annual impacts of the stronger imple- 
mentation sites climb to $1,141, as repotted in Chapter 4. 

4 The Jobs-Plus results are also noteworthy because they are averaged over the full target population in the 
developments, which included numerous residents who had only the most fleeting connection to the program, 
and even some who undoubtedly remained oblivious to it. In the other studies cited above, most sample mem- 
bers at least had to come forward and register for the program before they became part of the research, and 
many were subject to tangible financial penalties for failing to participate without good cause. Thus, it is not 
unreasonable to speculate that the impacts of Jobs-Plus on residents who actively sought the program's assis- 
tance may have been larger than the averages presented in this report for the entire target population. 
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suits for the stronger implementation sites were within the range of effects produced by the pro- 
grams in that study that had statistically significant impacts on earnings. 5 

The effectiveness of Jobs-Plus stands in marked contrast to the absence of labor market 
effects (at least in the short term) from an alternative approach that sought to improve public hous- 
ing residents’ self-sufficiency through residential mobility strategies — that is, by offering resi- 
dents special rent vouchers to subsidize their rent in the private housing market in low-poverty 
neighborhoods. Theoretically, the rent vouchers would allow residents to move to areas where 
more jobs are available and, perhaps, to become part of community-based social networks with a 
better flow of information about job opportunities and more supportive of work. As mentioned in 
Chapter 1, the U.S. Department of Housing and Urban Development’s (HUD's) Moving To Op- 
portunity demonstration tested this idea. This initiative aimed to achieve a broad range of quality- 
of-life improvements — including but not limited to better work-related outcomes — by helping 
public housing residents move to areas where poverty rates were less than 10 percent. The interim 
results have shown no impacts on residents’ employment or earnings within the first four years of 
follow-up. It may be that mobility strategies like these take more time to make a difference on 
employment, and perhaps the demonstration’s longer-term findings, when they become available, 
will reveal positive effects. Or it may be that mobility strategies are unlikely by themselves to pro- 
duce effects on work outcomes. If so, perhaps they would be more likely to produce such effects 
if, like Jobs-Plus, they included explicit assistance to help residents find, adapt to, or prepare for 
work, and extra financial incentives to do so. 

Can a voluntary employment program like Jobs-Plus be effective for 
public housing residents with very different backgrounds? 

Findings in Chapter 4 indicate that when Jobs-Plus was implemented relatively well, it 
markedly increased the earnings of many different subgroups of residents. For example, it sub- 
stantially increased the earnings of men as well as women, which belies the common perception 
that voluntary employment programs are not effective for men. In addition, Jobs-Plus produced 
large earnings gains for residents who were native-born, as well as for those who were immigrants 
from different parts of the world, including Mexico and Central America and Southeast Asia. 
Given the recent and projected growth of immigrant populations in the United States and the sali- 
ence of immigration in the current policy debate, this latter finding seems particularly relevant. 
Furthermore, Jobs-Plus was effective for public housing residents who were receiving welfare as 
well as for those who were not, although the program produced much larger impacts for the latter 
group. Thus, findings from the demonstration project provide important information for future 
employment programs in general, as well as for welfare-to-work programs in particular. 



5 Greenberg, Michalopoulos, and Robins, 2002. 
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Did Jobs-Plus contribute to the goals of welfare reform? 

Recognizing that many public housing residents were also welfare recipients, the de- 
signers of Jobs-Plus hoped that the program could contribute to the work-promoting objectives 
of welfare reform by helping such residents succeed in the labor market. How well did it do? As 
the subgroup analysis in Chapter 4 shows, Jobs-Plus did aid the cause of welfare reform by im- 
proving the earnings of residents who were welfare recipients at the time the program was 
launched. In the stronger implementation sites, it increased this subgroup’ s earnings during the 
four years after the rollout of Jobs-Plus by $761 per year — an 1 1 percent gain. 6 

How does this impact compare with estimates from studies unrelated to Jobs-Plus that 
have examined the consequences of welfare reform for public housing residents who were re- 
ceiving welfare? Although only a rough comparison is possible, the magnitude of Jobs-Plus’ s 
effect for the welfare subgroup is well within the range of a composite earnings impact estimate 
($719 per year, not inflation-adjusted) that is based on the results of six different random as- 
signment estimates of the earnings effects of welfare reform initiatives on public housing resi- 
dents. 7 It is important to recall from Chapter 3 that many residents of the comparison develop- 
ments in the Jobs-Plus evaluation similarly may have been involved in welfare reform interven- 
tions. If so, and if those interventions affected their earnings, one can speculate that Jobs-Plus’ s 
earnings impacts for the welfare subgroup reflected “added value” above and beyond any ef- 
fects of welfare reform itself. In other words, those who had access to Jobs-Plus’s package of 
rent incentives and on-site assistance and supports may have earned more than they otherwise 
would have earned as a result of the time limits, participation mandates, and financial incentives 
of welfare reform alone. 

These findings imply that Jobs-Plus offers welfare agencies a way to further an impor- 
tant part of their own mission — helping Temporary Assistance for Needy Families (TANF) 
recipients succeed in the labor market — for the portion of their population who live in public 
housing. More generally, the results suggest that welfare agencies may do well to align their 
own welfare-to-work efforts with the on-site strategies of public housing authorities. This could 
include colocating some of their own staff at the housing developments (even on a part-time 
basis), where they might be able to engage residents who are TANF recipients more easily and 
to interact with common purpose with other on-site staff. Welfare agencies might also consider 
requiring that their own frontline staff — whether located at the welfare agency or a housing 



6 Program impacts on welfare receipt rates and benefit levels were not estimated for subgroups of the study 
sample. However, in the three sites where data on welfare outcomes were available for the full sample, no im- 
pacts on these outcomes were evident. As noted, receipt of welfare fell dramatically over the course of the 
demonstration for both the program group and the comparison group. 

7 See Verma and Riccio, 2003, Table 6.1. The six relevant estimates in that table were converted to annual 
earnings impacts and averaged together, with each estimate weighted equally. 
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development — help market the work-based rent incentives already available to public housing 
TANF recipients under current federal housing law (see below), along with any other special 
work incentives that the local housing authority makes available. 

Would Jobs-Plus have been more effective if participation were 
mandatory? 

Among its recommendations for reforming the nation’s housing assistance programs, 
the federally appointed Millennial Housing Commission proposed in 2002 that, over time, eli- 
gibility for housing assistance be linked to residents’ compliance with work requirements — 
when appropriate employment services and supports are available. 8 This idea is inspired by wel- 
fare-to-work programs that make receipt of a full welfare grant conditional upon a parent's 
compliance with rules requiring him or her to work or prepare for work. Jobs-Plus had no such 
mandates per se. As a voluntary program, residents could ignore Jobs-Plus without worrying 
that they would put their rent subsidy at risk. 

It is impossible to say from the evidence available in this study whether Jobs-Plus 
would have been even more successful had it established a participation mandate or whether the 
often-substantial investment and administrative burden necessary to enforce such mandates 
would be worth the cost. However, it is noteworthy that, even in the absence of a formal work 
requirement linked to housing assistance, a large majority of residents across the six develop- 
ments (reaching 76 percent for the 2000 cohort) became formally attached to Jobs-Plus by en- 
rolling in the program or by living in a household participating in its rent incentives component. 
The resident survey data also showed that most household heads at the development — whether 
or not they were participating in Jobs-Plus — were involved in at least some activities related to 
employment and self-sufficiency at the time they were interviewed. For example, even in the 
absence of a mandate, 80 percent of the program group said that they were currently working or 
currently participating in a formal work-related activity (such as job search assistance, educa- 
tion, or training) or had looked for a job within the prior four weeks (Table 3.4). 

It is also important to understand that residents who were welfare recipients were sub- 
ject to their welfare department's TANF participation requirements. In fact, in order to encour- 



8 The Millennial Housing Commission's report states: “The MHC recommends that federal housing assis- 
tance programs encourage and facilitate expanded economic opportunity, recognizing that the working-age 
families living in assisted housing, like other able-bodied people, have an obligation to contribute to society as 
well as accept its help. Thus, MHC supports provision of the necessary services and supports to enable these 
families to find employment that will enable them to become self-sufficient and, when such services are avail- 
able, directly or through non-housing agencies, to accompany them with realistic work requirements. The Mil- 
lennial Housing Commission thus recommends that, over time, the housing assistance system require residents 
who are not elderly or disabled to work as a condition of receiving aid” (2002, p. 56). 
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age TANF recipients to join the Jobs-Plus program, nearly all the sites were able to forge an 
agreement with the local welfare departments to allow residents on welfare to satisfy their 
TANF obligations by participating in Jobs-Plus. 9 Consequently, such residents — if they chose 
this route — could be sanctioned (that is, have their grants reduced or terminated) by the wel- 
fare agency for failing to participate in Jobs-Plus. Moreover, according to federal rules, sanc- 
tioned recipients would not see their public housing rents reduced despite their lower income. 
Thus, at least for public housing residents who were TANF recipients, existing participation 
mandates that were imposed by the welfare system may have been a further inducement to take 
part in Jobs-Plus (in addition to the program’s rent incentives, on-site employment assistance, 
and other features). 

Finally, it is useful to recall that while Jobs-Plus in the stronger implementation sites 
had positive earnings impacts on the welfare subgroup, its effects were even larger for the non- 
welfare subgroup. The latter group, which faced no participation obligations, experienced aver- 
age earnings gains of $1,654 per year, representing an 18 percent increase. 

Did the Jobs-Plus rent incentives matter? 

For many years, proponents of helping public housing residents become more self- 
sufficient have claimed that it is critical to reduce the implicit tax imposed on residents’ earn- 
ings by federal policies that raise their rent when their income grows. The rent incentives cre- 
ated for Jobs-Plus are among the most ambitious attempts ever to change those policies. Did 
they work? Although it is not possible to answer this question with certainty, circumstantial evi- 
dence suggests that they did. 

Rent reform was one of a variety of innovations implemented as part of the overall 
Jobs- Plus package, and the evaluation’ s research design is not suitable for disentangling the in- 
dependent influence of each of the program’s features. However, patterns in the data suggest 
that the Jobs-Plus rent incentives were a crucial ingredient in the program’s effects on earn- 
ings. For example, the two sites where use of the rent incentives was lowest (Baltimore and 
Chattanooga) were the only sites to produce no impacts on earnings. In contrast, the first two 
sites to introduce the incentives (St. Paul and Seattle) showed the earliest evidence of earnings 
impacts. In addition, in one site (Dayton), where the 2000 cohort used the rent incentives at 
double the rate of the 1998 cohort (60 percent versus 30 percent), annual earnings impacts were 
noticeably larger for the later cohort ($1,189 versus $895). 

Qualitative data on residents’ reactions to the rent incentives are also important to con- 
sider. They suggest that the incentives may have had little to do with convincing persistently 



9 See Kato and Riccio, 2001. 
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nonemployed residents to begin working, perhaps because most residents had already shown an 
interest in working and many of those who were not working had major barriers — such as 
health problems — that rent incentives could not “fix.” Instead, incentives may have worked by 
encouraging those who were employed or who would have worked even without the incentives 
to hold onto their jobs longer or to seek reemployment quickly after losing a job. 10 

The circumstantial evidence that Jobs-Plus’ s rent incentives may have played a critical 
role in generating the program’s earnings impacts makes it plausible to conclude that the nar- 
rower incentives already available under current rent rules (established by the 1998 federal 
housing law) may also hold the potential to boost the earnings of qualifying residents. Even if 
the scope and depth of those existing incentives stayed the same, perhaps simply marketing 
them to residents more systematically and more aggressively might enhance their effectiveness, 
as the Jobs-Plus experience suggests. 

As important as they were, the rent incentives did not stand alone in Jobs-Plus. Despite 
their obvious value, the incentives needed to be marketed aggressively to residents. Educating 
residents about what the incentives entailed proved to be a formidable task that, to be done well, 
called on the collective efforts of the Jobs-Plus staff, resident outreach workers, community 
coaches, and housing authority property managers. But thanks to the communication and market- 
ing efforts, rent incentives became a critical “hook” that excited many residents about Jobs-Plus 
and drew them to the program. The lure of the incentives also resulted in many residents’ getting 
other forms of assistance, such as individualized job coaching, help with transportation costs, re- 
ferrals for other education or training or social services, and sometimes just ongoing informal en- 
couragement and support from the program. 11 Although it cannot be proved, the fact that the in- 
centives were heavily bound up with other program services may mean that the combination of 
elements offered by Jobs-Plus, not the incentives alone, may have been the main source of the 
program’s impacts (as postulated by the theory lying behind the multicomponent Jobs-Plus 
model). Findings from a number of welfare-to-work evaluations point to a similar conclusion — 
that incentives plus services may make a bigger difference than incentives alone. 12 



10 Gardenhire-Crooks, 2004. It is interesting to consider that, because of the quarterly nature of the UI data 
on which employment impacts were estimated, any effect that Jobs-Plus had with respect to shortening resi- 
dents’ spells of nonemployment from three months or more to less than three months could not be detected. 
However, such effects, if they did exist, would have contributed to the evaluation’s estimated impacts of Jobs- 
Plus on residents’ average earnings. This may be one reason why Jobs-Plus’ s earnings impacts are larger than 
its measured effects on employment rates. 

1 'Kato, 2003a; Liebow et al., 2004. 

12 In this regard, it is worth considering the findings of evaluations of two programs listed in Table 7.1: the 
Minnesota Family Investment Program (MFTP) and Canada’s Self-Sufficiency Project (SSP). Both provide evi- 
dence that financial incentives can promote higher earnings and employment retention. Yet, in the MF1P study, 
the incentives component had little independent effect. Overall, the evaluation found that MFTP was more effec- 

(continued) 
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Can Jobs-Plus help public housing developments become more mixed- 
income communities? 

The designers of Jobs-Plus envisioned the program not only as means for helping indi- 
vidual residents succeed in the labor market (knowing that many who benefited from it would 
leave public housing) but also as a strategy to help broaden the income mix within public hous- 
ing itself. The latter goal took on greater urgency with the advent of welfare reform in the mid- 
1990s. At that time, many housing authorities and policymakers were growing increasingly 
worried about the implications for public housing of imposing time limits on welfare benefit 
payments. They feared that, by reducing the welfare income of many of their residents, time 
limits would worsen the growing concentration of poverty and its attendant social problems in 
public housing. They also worried that residents would have more trouble paying their rent and 
that rent revenues would fall sharply. This would increase the amount of housing subsidies that 
those tenants would need, at just the time when federal operating subsidies — monies that HUD 
paid to housing authorities to cover the gap between total rent revenue and operating costs — 
were under increasing strain. Work offered a potential alternative. If residents could earn more 
money while living in public housing, they could eventually pay more of the true cost of their 
housing out of those earnings. This would help compensate for a loss of welfare income. Thus, 
it was hoped that by focusing on work, Jobs-Plus could, in time, help housing authorities ad- 
dress this looming financial threat. 

The development-level impact analysis presented in Chapter 5 shows that residents' earn- 
ings rose steadily in both the Jobs-Plus and the comparison developments even before Jobs-Plus 
began, thanks to the tail end of a booming economy and, perhaps, to other policy reforms. How- 
ever, in the stronger implementation sites, Jobs-Plus increased average earnings year to year 
within the developments to even higher levels — that is, beyond what those earnings would have 
been without the program. These impacts are important because they signal that a housing-based 
self-sufficiency initiative can help improve the earnings of existing tenants. This means that hous- 
ing authorities have more options than simply trying to recruit people who have jobs and higher 
incomes to move into public housing (which, itself, may be a considerable challenge). 



tive when it included the full package of incentives and welfare -to-work services and participation requirements 
(Miller et al., 2002). Results from the SSP evaluation indicate that the effects of its wage supplements on earnings 
were more enduring when they were combined with voluntary work-related services (Michalopoulos et al., 2002), 
although the incentives alone also had a large independent effect. These two sets of findings add weight to the 
conclusion that the Jobs-Plus incentives probably did make an important contribution to the program’s effective- 
ness and also that the incentives were probably not the only thing that mattered. 
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To be sure, the improvements caused by Jobs-Plus were not dramatic or transformative. 13 
In addition, this evaluation did not attempt to determine the net economic gain or loss to the hous- 
ing authority of operating a Jobs-Plus program (see below). Nonetheless, the program’s effects on 
development-level earnings were large, especially in sites with lower move-out rates, which saw 
the improvement in their tenants' earnings in some years exceed 20 percent. The results demon- 
strate that Jobs-Plus can at least play a supporting role in achieving the broader policy goal of 
building more mixed-income public housing, as called for by 1998 federal housing law. 

Can Jobs-Plus be effective if operated on a broader scale? 

The finding that Jobs-Plus worked in four different sites — Dayton, Los Angeles, St. 
Paul, and Seattle (until the residents were relocated under HOPE VI) — suggests that the Jobs- 
Plus model, if properly implemented, holds the potential to work in a variety of settings. Al- 
though the number of sites is small, they cover larger and smaller cities that also varied in the 
tightness of their housing and labor markets. The sites also served very different types of public 
housing tenants. As previously explained, these included a largely African-American single- 
mother population (Dayton), a heavily Latino and Southeast Asian population (Los Angeles), a 
population with a large number of Hmong refugees (St. Paul), and a highly ethnically diverse 
population in Seattle that included many immigrants and refugees from different East African 
nations as well as Southeast Asia. The evidence of positive effects across all these settings sug- 
gests that Jobs-Plus is a potentially “robust” intervention, not one that works only under highly 
selective circumstances or for a distinctive set of residents who are unusually predisposed to 
benefit from the program. The subgroup findings reinforce this conclusion, showing that Jobs- 
Plus had positive earnings impacts for a variety of tenant subgroups. These included residents 
with more prior employment and those with less prior employment, those with more prior wel- 
fare receipt and those with less prior welfare receipt, and those with longer tenures in public 
housing and those with briefer tenures. 

Given all the difficulties that the demonstration sites experienced in putting the Jobs- 
Plus model into practice, is it reasonable to expect that the model could be implemented prop- 
erly on a broader scale outside the context of a demonstration project? The evidence suggests 
that, in fact, it would be feasible, with the adequate funding and support, to replicate the model. 
Pour of the six Jobs-Plus sites were eventually able to get all program components into place, 
build functioning service partnerships with other agencies, implement fundamental rent reforms 
that helped make work pay, build a cadre of residents to reach out to others, and attract a major- 
ity of working-age residents to take advantage of at least some of what the program had to offer. 

13 There is also no evidence within either the program or the comparison developments over the period of 
data collection of a geometric rise that might suggest the occurrence of a “tipping -point’ ’ phenomenon, as hy- 
pothesized in Chapter 1 . 
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Of course, for the Jobs-Plus model to work on a broader scale, dedicated and qualified 
personnel are needed to staff it, like any good program. Although the program model is complex, 
it is not so complicated as to require staff with such specialized training that finding suitable can- 
didates would be unusually difficult outside the context of a special demonstration project. Indeed, 
it is reassuring that most of the staff h ir ed for Jobs-Plus at the successful demonstration sites were 
selected through the normal recruitment and hiring channels operated by the housing authorities 
and their key partners. It is also noteworthy that the service providers used by Jobs-Plus in the 
most effective sites were not of a difficult-to-replicate quality; rather, the program drew largely on 
existing services and service providers available in the local community. 

Still, the challenges of implementing the Jobs-Plus model should not be underestimated. 
The demonstration’s sponsors also made a considerable investment in technical assistance, 
which MDRC either provided directly (often with difficulty) or coordinated, to help the sites 
design and implement their programs. Many obstacles were encountered along the way to 
building solid programs. To some extent, these obstacles were the inevitable consequence of the 
“learning by doing” process inherent in new demonstration projects. The prospects for success- 
ful replication might thus be enhanced if new sites heed the many important lessons learned 
from the demonstration, which might help them avoid — or navigate better — some of the pit- 
falls or challenges that they are likely to encounter along the way . 14 Replication might be further 
enhanced if a qualified intermediary organization or HUD itself were available to provide tech- 
nical assistance in this process. In addition, housing authorities hoping to replicate Jobs-Plus 
might also find it advantageous to subcontract the day-to-day, on-site operational responsibili- 
ties to an experienced employment and training provider rather than build and staff a new pro- 
gram entirely from scratch. Still, housing management staff would need to maintain a close op- 
erational partnership with the program, as suggested next. 

What would the replication of Jobs-Plus require of housing authorities? 

It goes almost without saying that the housing authority’s sustained commitment to the 
program, as part of a broader collaborative effort, is critical to any successful replication of 
Jobs-Plus. Although Jobs-Plus was originally envisioned as a joint endeavor of local inter- 
agency and resident collaboratives, the housing authority must provide the glue that keeps such 
partnerships together and that supervises and holds the Jobs-Plus staff accountable for high per- 
formance . 15 Other agencies may help fund, support, and provide critical services to the program, 
but they are unlikely to want to assume the kind of shared governance role over Jobs-Plus that 



14 These lessons have been described in MDRC’s earlier reports on the implementation of Jobs-Plus. See, 
for example. Bloom, 2000; Gardenhire-Crooks, 2004; Kato, 2003a, 2003b, 2004; Kato and Riccio, 2001; Lie- 
bow, 2004; Miller and Riccio, 2002; and Riccio, 1999. 
l5 Kato and Riccio, 2001. 
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was originally attempted in the collaboratives. Thus, more streamlined collaborative relation- 
ships and decision-making strategies than were attempted in the demonstration, with strong 
leadership and accountability provided by the housing authority, may be advantageous in any 
replication effort. Moreover, the commitment of senior housing authority officials outside a 
special demonstration project would undoubtedly be enhanced if federal laws and regulations 
required that housing management assume responsibilities for ensuring that residents have ac- 
cess to and take advantage of work-related services — and if appropriate funds were available 
to make these added responsibilities feasible to embrace. 

Assistance from the housing management staff is critical to the day-to-day operations of 
the program. In particular, replication would be enhanced when housing management staff are 
partners with Jobs-Plus in marketing the rent incentives and promoting a strong message about 
work and about how the program can help residents succeed in the labor market. Indeed, the 
housing authority and its property management staff could publicize to tenants a message that 
assistance with employment “comes with tenancy’' in public housing. This would represent a 
commitment to a broader mission for housing authorities beyond their already vital one of pro- 
viding decent, well-managed housing. But it is also a mission that would be feasible for them to 
take on, especially with the right community partners and funding. 

Housing authorities seeking to replicate Jobs-Plus would also do well to involve resi- 
dent representatives in planning and operating Jobs-Plus. Residents can help establish the le- 
gitimacy of the program by helping agency-based planners understand what may or may not 
appeal to their fellow tenants. Using residents as outreach workers (for instance, as part of a 
community support for work component or as regular paid staff) can help Jobs-Plus become a 
pervasive and widely known source of employment support throughout a housing development. 
At the same time, it is important not to let worthy support for resident “empowerment” result in 
assigning critical program management or line staff functions (for example, case management 
roles) to residents who are not adequately prepared or trained for those responsibilities, as oc- 
curred early on in some Jobs-Plus sites. 

What does it cost to operate a Jobs-Plus program? 

Although the Jobs-Plus evaluation did not include a detailed cost analysis, it is possible 
to put the cost of operating Jobs-Plus into some perspective, showing the scale of investment 
that this type of intervention may require on the part of housing authorities and their partner 
agencies . 16 (Recall that the demonstration’s sponsors viewed Jobs-Plus as more than just a hous- 
ing authority and HUD initiative and that, in keeping with the principle of collaboration, the 



16 These estimates are based on budget information submitted to MDRC by the Jobs-Plus sites. 
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local welfare and workforce agencies — the target groups of which include many public hous- 
ing residents — would share the burden of funding it.) 

Drawing on the experience of the three stronger implementation sites (which came 
closest to implementing the full vision of Jobs-Plus), one very rough estimate suggests that the 
immediate budgetary cost of operating the on-site features of the program (including the rent 
incentives) could, under some circumstances, amount to approximately $150 per targeted resi- 
dent in any given month. If all else were equal, this would imply that a housing authority with, 
say, 250 eligible, working-age residents may need an annual budget in the vicinity of $450,000 
per year to provide the on-site services and rent incentives — although, for reasons discussed 
below, this may represent a high-end estimate. Note that about 35 percent ($158,000) of the es- 
timated annual budget would be spent on rent incentives, while about 65 percent ($292,000) 
would be required for all other budgeted expenditures associated with the program. 17 

The actual added cost to government may, however, be considerably lower. Chapter 3 
notes that, even in the absence of Jobs-Plus, housing authorities would have spent money on 
alternative self-sufficiency services and rent incentives, as evidenced by the fact that they did so 
for numerous residents of the comparison developments in the demonstration sites. Conse- 
quently, the incremental investment necessary to operate the on-site features of Jobs-Plus would 
be less than the above gross cost estimate. How much less would depend on what other em- 
ployment-related services and incentives were already being provided. For purposes of illustra- 
tion, suppose that the combination of alternative services and rent incentives under the 1998 
housing law cost about 20 percent of the investment made for Jobs-Plus and that these expendi- 
tures (or the services they provided) would be redirected to Jobs-Plus. 18 That would mean that 
the extra on-site cost of Jobs-Plus would be about $120 per resident per month in any given 
month, yielding a total annual program budget of roughly $360,000. If the cost of alternative 
services amounted to 40 percent of Jobs-Plus costs, then the net additional investment for Jobs- 
Plus would fall to $90 per targeted resident per month, for a total budget of $270,000 per year. 

A number of factors could make the on-site operating costs of a Jobs-Plus program higher 
or lower than those estimated here. One crucial driver of costs is a development’ s move-out rate. 
The costs of serving newer tenants is likely to exceed the costs of serving other tenants, assuming 

17 It is not possible with the data available for this evaluation to estimate the total average cost over time of 
providing on-site services and rent incentives specifically to members of the 1998 cohort for comparison with 
the program’s estimated impact on that group. 

ls By way of comparison, data from a comprehensive evaluation of 10 welfare-to-work programs across 
the country indicate that government expenditures on services per person in the control group equaled 5 1 per- 
cent of the expenditures per program group member (see Hamilton et al., 2001, Table 13.4). Also, by way of 
comparison, the survey data in Table 3.4 indicate that, across the five survey sites, the comparison group's rate 
of participation in services obtained through the housing authority or at the housing developments was 44 per- 
cent of the program group's rate (13.4/30.5 = 43.9). 
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that, in general, residents who stay will eventually reduce their involvement in the program. Con- 
sequently, other things being equal, the sustained costs of operating a Jobs-Plus program are likely 
to be lower where resident turnover is lower. Costs might also be lower if the program were oper- 
ated more efficiently as an ongoing program than it was as a new demonstration project with a 
protracted design and trial-and-error period or if the program were operated across more devel- 
opments in a city, thereby creating substantial administrative economies of scale. 

Is Jobs-Plus a good financial investment? 

This question cannot be answered by the present evaluation, which was not designed 
to include a benefit-cost analysis. Such an analysis would have allowed a comparison of Jobs- 
Plus net impacts — for example, its value added — to its net costs. However, as noted above, 
only rough estimates of gross costs are available, and readers should resist the temptation to 
compare gross costs (“apples”) to net impacts (“oranges”). As a result, it is impossible to cal- 
culate a rate of return on the government’s investment (or on the specific investments made 
by housing authorities — using HUD funds and rent revenues — and their welfare and work- 
force agency partners). 19 

For the same reason, it is not possible to estimate the precise dollar value of the net eco- 
nomic gain or loss to residents in the Jobs-Plus study sample. However, based on the findings in 
Chapter 4, it is reasonable to expect that, over time, residents in the Jobs-Plus developments 
came out ahead financially. 

Recall that Jobs-Plus’ s positive effects on residents' earnings were substantial and endur- 
ing, even after many residents moved out of public housing (at which point the program’s on-site 
investment in them ended). For example, in the three stronger implementation sites, Jobs-Plus had 
positive earnings impacts during each of the four years following the rollout of the program. 
These effects averaged $1,141 per resident per year, accumulating to $4,563 per resident over the 
four-year period. (It is important to remember that these estimates are averaged over all targeted 
residents, whether or not they participated in Jobs-Plus and whether or not they worked. Hence, 
the effects on those who became involved in the program and worked are undoubtedly larger.) 

Trends in the program’s earnings impacts also indicate that Jobs-Plus’ s effects continued 
past the end of data collection. Furthermore, Jobs-Plus did not produce any reductions in resi- 



19 To measure this return on investment would require knowing, among other things, (1) the costs of in- 
kind services and activities provided by local housing authorities and their partner organizations; (2) the costs 
of employment-related services provided to sample members by organizations that were not part of Jobs-Plus 
and thus outside of its accounting system; (3) the costs of income transfer payments (for example, TANF, food 
stamps, and EITCs) made by government organizations to sample members; and (4) the effects of Jobs-Plus on 
earnings for a number of years beyond the present follow-up period. 
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dents’ welfare income, and many residents benefited from the extra money they obtained through 
the program’s rent incentives. For example, data show that, over the first one-and-a-half to two 
years of participation in the rent incentives program, households in the 1998 cohort in the three 
stronger implementation sites saved more than $170 per month in rent. 20 (This equates to over 
$2,000, on average, for households participating in the program for a year.) Some Jobs-Plus par- 
ticipants may have also benefited from increased use of the EITC, and many got program help 
with work-related costs (for example, transportation and child care). 21 All these factors support a 
conclusion that Jobs-Plus was likely to be financially beneficial from the perspective of residents. 

Are the lessons of Jobs-Plus relevant outside public housing? 

Many lessons from Jobs-Plus could help efforts to improve the labor market experi- 
ences of low-income populations who are not living in public housing. For example, although 
the place-based elements of Jobs-Plus would not be directly relevant, the sites’ experience with 
implementing new forms of rent-based work incentives and combining these with services fo- 
cused on work may hold lessons relevant to strengthening the operation of HUD’ s Family Self- 
Sufficiency program, which is largely directed toward recipients of portable Section 8 rent 
vouchers, which subsidize rents in the private housing market. In addition, some of sites’ day- 
to-day operating experiences — for example, in coordinating employment-related services 
through the housing authority in cooperation with other local agencies and in recruiting highly 
diverse groups of people into a voluntary employment program connected with the housing au- 
thority — may also be informative for that program. 

For a broader range of community initiatives, which are extremely difficult to evaluate, 
the findings of Jobs-Plus are important because they offer rare and convincing evidence that a 
place-based intervention can improve the earnings of very low-income residents under certain 
conditions. At the same time, the Jobs-Plus findings caution that the degree to which individual- 
level effects on employment outcomes will translate into community-level effects will depend 
to an important extent on residents’ move-out rates. Furthermore, it may take much bigger ef- 
fects on employment and earnings rates than were observed in Jobs-Plus developments (on top 
of the more striking improvements that had already occurred) in order for such effects to lead to 



20 These estimates are based on data collected from housing authority records available through the end of 
2002 for households taking part in the rent incentives program. The savings use traditional HUD rent rules 
(which calculate rent as 30 percent of adjusted income) as a benchmark. By site, the participating households 
saved the following amounts in rent per month, over the following average number of months: Dayton, $228 
over 20 months; Los Angeles, $96 over 19 months; and St. Paul, $192 over 27 months. These estimates do not 
include any additional savings that the households may have enjoyed from other features of the Jobs-Plus in- 
centives plans in some sites, such as rent credits for employment retention. 

2 ’For detailed examples of how residents can improve their net income after rent by taking advantage of other 
available financial work supports in addition to the Jobs-Plus rent incentives, see Miller and Riccio (2002). 
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improvements in other dimensions of the community or neighborhood quality of life. Because 
the truly big changes in employment rates took place before Jobs-Plus began (and, hence, prior 
to the administration of the study’s baseline survey), this evaluation does not offer a good test of 
whether big employment increases in a community can spark very large improvements in the 
quality of neighborhood life. 

Finally, Jobs-Plus holds many practical lessons for constructing and operating labor 
market interventions. In particular, the experiences of the six sites speak directly to the chal- 
lenges and opportunities of using “places” as the focus of and platform for a work-promoting 
intervention, including the opportunities to interact with and influence residents informally, 
where they live, in contrast to the more typical “by-appointment” staff-participant interactions 
in many employment programs. The sites’ experiences also point to productive strategies (and 
cautions about unproductive approaches) for building partnerships among multiple agencies — 
including welfare, workforce, and social services agencies — to address the employment needs 
of low-income populations and for involving local residents in that process. 



167 




Appendix A 

Supplementary Tables for Chapter 2 




The Jobs-Plus Demonstration 

Appendix Table A.l 



Selected Population Characteristics of the Census Tracts 
in Which the Jobs-Plus and Comparison Developments Are Located (2000 Census) 





Baltimore 


Chattanooga 


Dayton 




Los Angeles 


St. Paul 


Seattle 




Jobs- 




Jobs- 




Jobs- 




Jobs- 




Jobs- 




Jobs- 




Characteristic 


Plus Comparison 


Plus Comparison 


Plus Comparison 


Plus Comparison 


Plus Comparison 


Plus Comparison 


Race/ethnicity (%) 


























Black, non-Hispanic 


98 


83 


96 


81 


98 


71 


1 


5 


22 


18 


6 


26 


White, non-Hispanic 


1 


10 


2 


15 


1 


25 


2 


4 


13 


35 


82 


35 


Hispanic 


0 


2 


1 


3 


0 


1 


46 


87 


5 


6 


6 


7 


Asian 


0 


4 


0 


1 


0 


0 


49 


2 


62 


37 


8 


25 


Foreign born (%) 


1 


5 


1 


3 


0 


0 


62 


40 


48 


28 


26 


25 


Households headed by a 
single parent (%) 


36 


23 


46 


18 


29 


33 


22 


26 


27 


24 


16 


9 


High school graduates, 
among persons age 25 or 
older (%) 


45 


53 


55 


54 


63 


64 


34 


35 


49 


64 


77 


70 


Median family income 11 ($) 


19,071 


16,532 


10,451 


18,420 


20,565 


14,516 


20,821 


19.500 


20,219 


22,141 


51,307 


31,472 


Poverty rate (%) 


43 


51 


62 


46 


41 


49 


37 


41 


54 


43 


16 


33 


Unemployment rate (%) 


21 


16 


20 


13 


23 


11 


19 


8 


8 


5 


4 


7 


Total population in each tract 


3,363 


3,401 


2,431 


8.022 


3,169 


3,511 


3,276 


3,262 


1,918 


5,581 


5,943 


9,687 



SOURCES: MDRC calculations using data from the U.S. Bureau of the Census, Census 2000 Summary Files 1 and 3. 
NOTE: '‘Dollar amounts are not inflation-adjusted. 
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Appendix Table A.2 

Selected Housing Characteristics of the Census Tracts 
in Which the Jobs-Plus and Comparison Developments Are Located (2000 Census) 





Baltimore 


Chattanooga 


Dayton 




Los Angeles 


St. Paul 


Seattle 




Characteristic 


Jobs- 

Plus Comparison 


Jobs- 

Plus Comparison 


Jobs- 

Plus Comparison 


Jobs- 

Plus Comparison 


Jobs- 

Plus Comparison 


Jobs- 

Plus Comparison 


Rental vacancy rate (%) 


16 


3 


3 


9 


14 


24 


3 


2 


0 


1 


5 


5 


Median household gross 
rent ($) 


255 


199 


207 


234 


188 


310 


435 


360 


197 


501 


566 


505 


Fair market rent 
(FMR)“ ($) 


643 


643 


513 


513 


549 


549 


766 


766 


684 


684 


772 


772 


Homeownership 
rate b (%) 


31 


12 


24 


14 


42 


22 


8 


25 


1 


38 


66 


28 


Median value of owner- 
occupied houses ($) 


45,800 


NA 


52,900 


58,600 


58,000 44,550 


142,900 


144,900 


NA 


80,600 


218,900 


NA 


Housing stability 1- (%) 


58 


49 


61 


47 


58 


45 


62 


54 


54 


51 


48 


33 


Total population in 
each tract 


3,363 


3.401 


2,431 


8,022 


3,169 


3,511 


3,276 


3,262 


1,918 


5,581 


5,943 


9,687 



SOURCES: U.S. Department of Housing and Urban Development, http://www.huduser.org/datasets/fmr/FMRHIST.xls, data extracted on October 4, 2002; 
Census 2000 Summary Files 1 and 3; and MDRC calculations. 

NOTES: J The FMRs are gross rent estimates; they include shelter rent and the cost of utilities, except telephone. FMRs listed are for two-bedroom 
apartments. The geographical unit they cover is the metropolitan area, so the FMR is the same in the Jobs-Plus and comparison developments in each city. 
b This is the rate of owner-occupied housing. 

^Housing stability was calculated by dividing the number of people living in a MSA/PMSA in the same house five years earlier by the total census tract 
population for the year 2000. 
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Selected Characteristics of Targeted Residents Aged 21 to 61 
Living in the Jobs-Plus Development or Its Comparison Development(s) in 1998, by Site 





Baltimore 




Chattanooga 




Dayton 


Los Angeles 




St. Paul 




Seattle 


Characteristic 


Jobs- 

Plus Comparison 


Jobs- 

Plus 


Comparison 


Jobs- 

Plus 


Comparison 


Jobs- 

Plus 


Comparison 


Jobs- 

Plus 


Comparison 


Jobs- 

Plus 


Comparison 


Race/ethnicity (%) 


White (non-Hispanic) 


0 


1 


4 


3 


1 


9 


1 


2 


3 


9 


8 


7 


Black (non-Hispanic) 


99 


99 


94 


95 


97 


89 


6 


9 


21 


24 


32 


33 


Hispanic 


1 


0 


2 


2 


0 


0 


79 


81 


4 


2 


0 


3 


Asian/Pacific Islander 


0 


0 


0 


0 


0 


0 


12 


2 


56 


61 


28 


38 


Other 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1 


14 


8 


Missing 


0 


0 


0 


0 


2 


1 


1 


7 


16 


3 


19 


11 


Household head (%) 


91 


88 


93 


93 


93 


91 


57 


67 


66 


73 


70 


66 


Female (%) 


81 


83 


91 


90 


86 


81 


65 


70 


66 


70 


71 


73 


Age (%) 


21-24 years 


13 


11 


27 


23 


26 


27 


12 


10 


13 


18 


12 


13 


25-34 years 


33 


33 


38 


39 


42 


36 


28 


33 


40 


42 


33 


29 


35-61 years 


55 


56 


35 


38 


32 


37 


59 


58 


47 


39 


55 


58 


Average age (years) 


37 


37 


32 


33 


32 


32 


39 


38 


36 


33 


37 


37 


Lived in a household 
with (%): 


Two or more adults 


16 


24 


15 


14 


14 


19 


74 


62 


67 


52 


54 


54 


No children 


45 


40 


17 


26 


27 


28 


26 


21 


5 


9 


23 


27 


Children ages 


0-5 years 


25 


26 


44 


43 


47 


43 


29 


35 


56 


56 


42 


35 


6-17 years 


44 


51 


64 


51 


48 


50 


63 


72 


88 


69 


62 


56 


Sample size 


367 


649 


282 


513 


351 


608 


513 


443 


312 


245 


434 


349 



(continued) 
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SOURCE: MDRC calculations using data from public housing authority records. 

NOTES: This sample includes all residents of each public housing development in October 1998 who were between 21 and 61 years old and not listed as 
disabled on public housing authority records. These residents are referred to as the "1998 cohort" in this report. 

Because of missing data on these records, the sample size for each characteristic may vary. 

In Baltimore, Chattanooga, and Dayton, which had two comparison developments each, the numbers presented here represent equally weighted averages 
for both developments. 
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Appendix Table B.l 



Baltimore: 

Comparison of Local Context, Services, and Incentives at the Jobs-Plus Development and Its Comparison Developments 



Program Feature 


Jobs-Plus Development 

Gilmor Homes 


Comparison Developments 

Perkins Homes and Somerset Courts 


Housing and 
Neighborhood 


Development includes 571 units in low-rise buildings in Sand- 
town-Winchester neighborhood of West Baltimore. City’s most 
distressed area. High incidence of property and violent crimes, 
including drug trafficking. Resident drug use major barrier to 
employment. Few neighborhood jobs. BWI Airport jobs as shut- 
tle drivers and security personnel dried up after 9/11. 


Development includes 688 and 257 units in Perkins and Somer- 
set, respectively. Both are low-rise buildings in East Baltimore, 
surrounded by extensive real estate development, tourism, and 
business investment. Perkins Homes in East Harbor Empower- 
ment Zone, near Oriole Park at Camden Yards, Johns Hopkins 
and Mercy Hospitals. Somerset Courts is walking distance to 
Inner Harbor. Drugs and crime a bigger problem in Perkins than 
Somerset. 


Demographics 


Predominantly African-American single mothers as household 
heads. Sizable minority of disabled and seniors. 


Predominantly African-American single mothers as household 
heads. 
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Appendix Table B.l (continued) 



Employment-Related 

Services 


On-site individualized intake, assessment, and case man- 
agement. Job search and job readiness assistance are avail- 
able either from Jobs-Plus staff or through referrals to such 
partner agencies as Eden Jobs and Goodwill Industries. 
Until January 2002, Jobs-Plus offered on-site driver’s edu- 
cation linked to the auto-purchasing program of a partner 
agency. A satellite office of the Vision for Health Consor- 
tium (VHC) of public and private health organizations of- 
fered on-site health assessments and substance abuse treat- 
ment referrals until it closed, in September 2002. TANF 
recipients had an on-site caseworker from Work Matters — 
a welfare-to-work program of the local housing authority 
and the WIA agency — until June 2001, when the case- 
worker was reassigned to a downtown office. TANF recipi- 
ents were redirected to downtown welfare offices (and 
away from Jobs-Plus) for employment services. 


Only one on-site social worker per development. Employment 
assistance through off-site referrals. Housing authority- 
sponsored training and job programs (construction, lawn care, 
security) require drug and alcohol screenings and GED, limit- 
ing participation. 


Rent Incentives 


Implemented in November 2000, the rent incentives plan 
at Gilmor Homes reduces the percentage of adjusted in- 
come that is used to calculate working families’ rent (from 
the traditional 30 percent to 20 percent). Half the reduced 
rent is deposited in a non-interest-bearing escrow account 
for each month that residents work during a consecutive 
12-month period. At the end of each annual cycle, the sav- 
ings are rebated to residents. However, residents who were 
not employed for 30 days or more during any 12-month 
cycle forfeited the savings accumulated during that period. 


Began implementing the rent incentives of the Quality Housing 
and Work Responsibility Act (QHWRA) in late 2003. 


Community Support for 
Work 


hi 2001, Jobs-Plus trained residents to be court captains, 
who help circulate information about services and job open- 
ings and recruit participants for such activities as the on-site 
driver’s education class, a workshop on the Earned Income 
Tax Credit (EITC), and the rent incentives program. 


No formal efforts to engage residents in promoting employ- 
ment services and job opportunities to other residents. 
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Chattanooga: 

Comparison of Local Context, Services, and Incentives at the Jobs-Plus Development and Its Comparison Developments 



Program Feature 


Jobs-Plus Development 

Harriet Tubman Homes 


Comparison Developments 

College Hill Courts and Emma Wheeler Homes 


Housing and 
Neighborhood 


Development includes 423 low-rise apartment units in East Chat- 
tanooga, 3 miles northeast of downtown Chattanooga. Located 
near train yard and industrial area. Since August 2002, all Chatta- 
nooga public housing developments under private management. 


College Hill Courts includes 497 low-rise apartment units; Emma 
Wheeler Homes includes 340 units. College Hill Courts in West 
Chattanooga, near downtown Chattanooga and adjacent to several 
public transportation lines and major roadways. Emma Wheeler 
Homes farther away from downtown. In 2001, temporary reloca- 
tion of Emma Wheeler Homes residents under way for HOPE VI 
renovation. 


Demographics 


Predominantly African-American single mothers as household 
heads, along with 100 units for elderly and disabled. Beginning 
first quarter 2001, influx of McCallie Homes residents displaced 
by HOPE VI renovations, who were described as “hard to place” 
in jobs. 


Predominantly African-American single mothers as household 
heads. 



(continued) 
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Employment- 
Related Services 


Jobs-Plus was reconstituted between June 2000 and June 2002, in 
attempt to improve service delivery. Offered residents on-site in- 
take, assessment, and job readiness assistance. Those who were 
deemed job-ready were referred to a job coach, who divided her 
time between Jobs-Plus and the Southeast Tennessee Career Cen- 
ter, where she helped residents use job search services. Education 
and training as well as support services available, largely through 
referrals to off-site providers. However, Hamilton County Board of 
Education provided on-site GED preparation and computer literacy 
classes, and Academy of Allied Health offered several cycles of 
certified nurse assistant training. Other on-site programs included 
Family Neighborhood Center, which provided job training, a food 
pantry, and after-school programs. Enduring operational difficul- 
ties led to a termination of formal Jobs-Plus services in June 2002. 


Emma Wheeler Homes had after-school tutorial program in aging 
YMCA community center, and adult GED and computer classes at 
elementary school next door. In contrast. College Hill Courts had 
extensive multiagency resource center under management of 
Westside Community Development Corporation, with employ- 
ment, training, education, and social services (including child care, 
health clinic, and 12-step residential drug treatment facility), and 
resident-operated catering service and coin-operated laundry. Ad- 
ditional health/fitness center built in 2002. 


Rent Incentives 


Implemented in November 2000, Chattanooga’s financial incen- 
tives plan involves a two-step rent schedule for households with 
working members. Step 1 calculates rents at 10 percent of count- 
able income (for 16 months), and Step 2 calculates rents at 20 per- 
cent (for the remaining time of the demonstration). This is the only 
Jobs-Plus component that has been formally offered in Chatta- 
nooga since June 2002. 


Began implementing QHWRA rent incentives in 2000. 


Community 
Support for Work 


Jobs-Plus became a financial-incentives-only program before it 
could fully implement this component. 


No formal efforts to engage residents in promoting employment 
services and job opportunities to other residents. 
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Appendix Table B.3 
Dayton: 

Comparison of Local Context, Services, and Incentives at the Jobs-Plus Development and Its Comparison Developments 



Program Feature 


Jobs-Plus Development 

DeSoto Bass Courts 


Comparison Developments 

Arlington Courts and Parkside Hornes 


Housing and 
Neighborhood 


Development had 382 row-house apartment units about 4 miles 
southwest of downtown Dayton in low-income residential neighbor- 
hood. ( 128 units of original development demolished in mid-2002 to 
increase open space.) Few employment opportunities in walking dis- 
tance. Crime and substance abuse significant problems. 


Arlington Courts includes 314 row-house apartment units. Parkside 
Homes includes 518 units. Arlington Courts in West Dayton, Park- 
side Homes in North Dayton toward airport. Both in low-income 
residential areas, although Arlington Courts in proximity to retail 
shopping center, groceiy store, and fast-food establishments. Crime 
and substance abuse significant problems. 


Demographics 


Predominantly African-American single mothers as household 
heads. 


Predominantly African-American single mothers as household 
heads. Parkside Homes includes Caucasian residents from Appala- 
chian areas. 



(continued) 
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Employment- 
Related Services 


Jobs-Plus offers residents on-site intake, assessment, case manage- 
ment, and job readiness and job search assistance as well as job 
retention follow-up. To conduct independent job search, residents 
have on-line access at Jobs-Plus to the Job Center's database of job 
openings, and they are referred to the Job Center and to off-site 
partners for most education, training, and support services. Jobs- 
Plus has also hosted several cycles of on-site job trainings, including 
cash-register training sponsored by Walgreen' s drugstore chain and 
workshops on household management and income tax preparation. 
In 2002, Jobs-Plus partnered with RETS Tech to recruit participants 
for its heating, ventilation, air-conditioning, and refrigeration 
(HVACR) training program, and it provided case management fol- 
low-up to ensure training completion. The program also received 
housing authority funding to offer employment and substance abuse 
assistance to noncustodial fathers of children who live at DeSoto 
Bass Courts, even when the fathers were not on the lease. 


No sizable community centers or on-site employment services at 
Arlington Courts and Parkside Homes. Caseworkers of Sankofa, 
the nonprofit arm of housing authority entrusted with resident 
services, located off-site at central office. Some after-school activi- 
ties on-site for children and youth. 


Rent Incentives 


Implemented in May 2000, Dayton's financial incentives plan 
eliminates income-based rent calculations and replaces them with 
a two-step, flat-rent approach. Jobs-Plus’ s flat rents are set at a rate 
lower than what most households with full-time workers would 
pay if their rent remained income-based. During Step 1, which 
begins on enrollment, rents are set for one year at about one-third 
the normal market-based flat rent for a given unit size. During 
Step 2, rent increases are limited to about one-half the normal flat 
rent for a similar unit, for the remainder of the demonstration. 


Began implementing QHWRA rent incentives in 2000. 


Community 
Support for Work 


Jobs-Plus in Dayton trained residents to be building captains, who 
distribute information about the program’s services and about job 
openings and who recruit participants. Jobs developer and other 
staff also made efforts to reach residents with employment infor- 
mation and assistance at community events, such as the well- 
attended basketball tournaments and family picnics organized in 
partnership with the community-based Men of Standards. 


No formal efforts to engage residents in promoting employment 
services and job opportunities to other residents. 
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Appendix Table B.4 
Los Angeles: 

Comparison of Local Context, Services, and Incentives at the Jobs-Plus Development and Its Comparison Development 



Program Feature 


Jobs-Plus Development 

William Mead Homes 


Comparison Development 

Dana Strand Village 


Housing and 
Neighborhood 


Development includes 414 low-rise apartments in Boyle Heights 
neighborhood of East Los Angeles. Older but well-maintained. 
Downtown Los Angeles to north and Chinatown southwest. 
County prison nearby. Surrounded by light industrial area with 
poultry processing plant, UPS warehouse, and wholesale food and 
Asian import distributors. Problems with substance abuse, drug 
dealing, other crimes, and gangs somewhat less than in comparison 
development. 


Development consisted of 384 low-rise housing units. Between 
May 2002 and August 2003, households gradually moved off-site 
to make way for HOPE VI construction. In residential Wilmington 
neighborhood of South Bay area. Proximity to airports, Los Ange- 
les and Long Beach ports, petroleum refineries, beaches, amuse- 
ment parks, and shopping malls, with jobs in food/drinking estab- 
lishments, construction, janitorial services, warehouses, retail 
stores, repair shops, laundries, medical services, security firms, and 
airport shuttle services. Problems with substance abuse, drug deal- 
ing, other crimes, and gangs. 


Demographics 


Predominantly Hispanics of Mexican and Central American origin, 
with minority of Southeast Asians and blacks. Many households 
with two or more adults including male household heads. Immi- 
grant-related barriers to employment include limited English profi- 
ciency and, for some, lack of legal right to work in the United States. 


Predominantly Hispanics of Mexican and Centr al American origin. 
Many households with two or more adults including male house- 
hold heads. Immigrant-related barriers to employment include lim- 
ited English proficiency and, for some, lack of legal right to work 
in the United States. 
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Employment- 
Related Services 


On-site job preparation and job search assistance are available for 
individuals and groups. The housing authority designated Jobs- 
Plus as the “portal” for its Career Service Centers (CSCs), and 
Jobs-Plus publicized CSC services and boosted take-up by help- 
ing residents complete applications for CSC education and train- 
ing opportunities. Collaborative partners help Jobs-Plus provide 
services through collocated staff or off-site referrals. On-site at 
both housing developments are an employment caseworker from 
the welfare agency and a job developer from the Employment 
Development Department. East Los Angeles Women’s Center 
provides domestic violence counseling, and the East Los Angeles 
Skills Center offers GED classes for Spanish speakers. 


Few on-site services. Computer Learning Center (CLC) programs 
in self-directed keyboarding training, GED and ESL software 
geared toward youth. Support groups for adults dealing with sub- 
stance abuse or domestic violence. HOPE VI support services 
primarily for helping residents file Section 8 paperwork and find 
alternate housing. 


Rent Incentives 


Implemented in June 2000, Phase 1 of Los Angeles’s plan either 
froze the rent of participating households for 18 months (if the 
current rent was less than the Jobs-Plus flat rent) or reduced it to 
the proposed flat rent (if the current rent was higher than the flat 
rent). During Phase 2, which began in February 2002, participat- 
ing households paid the flat rent. 


Began implementing QHWRA rent incentives in 2001. 


Community 
Support for Work 


Beginning in November 2000, Jobs-Plus recruited approximately 
ten community coaches and trained them on economic develop- 
ment and community organizing to help circulate Jobs-Plus in- 
formation and recruit program participants. Coaches subsequently 
played a key role in developing services, including on-site GED 
classes for Spanish speakers. Coaches also work closely with their 
resident councils to sponsor community events. Of all the sites, 
Los Angeles was most successful in implementing this Jobs-Plus 
component. Strong commitment to resident empowerment 
through leadership development and civic participation. 


No formal efforts to engage residents in promoting employment 
services and job opportunities to other residents. 
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Appendix Table B.5 
St. Paul: 

Comparison of Local Context, Services, and Incentives at the Jobs-Plus Development and Its Comparison Development 



Program Feature 


Jobs-Plus Development 

Ml. Airy Homes 


Comparison Development 

Roosevelt Homes 


Housing and 
Neighborhood 


Development includes 298 suburban townhouses, renovated 1993 
to 1996, and 152 senior high-rise units. Safe, well guarded by 
housing authority’s police unit. In Thomas-Dale neighborhood, 
near hospital and medical complex, downtown St. Paul busi- 
nesses, and state and city government offices. 


Development includes 314 suburban townhouse units. Indistin- 
guishable from middle-class residential surroundings in Greater 
Eastside neighborhood. Safe, well guarded by housing authority’s 
police unit. 


Demographics 


Predominantly residents of Southeast Asian origin, mostly 
Hmong. Recent influx of East African and Mexican immigrants. 
Language- and immigrant-related barriers to outreach, service 
delivery, and employment, such as limited English proficiency 
and post-traumatic stress disorder. 


Predominantly residents of Southeast Asian origin, mostly 
Hmong, although more African-American residents than Mt. Aiiy 
Homes. Recent influx of East African and Mexican immigrants. 
Language- and immigrant-related barriers to outreach, service 
delivery, and employment, such as limited English proficiency 
and post-traumatic stress disorder. 
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Employment- 
Related Services 


Spacious community center for program offices and activities. On- 
site assistance with job readiness, job search, and job retention. 

ESL and U.S. citizenship classes, GED instruction. Head Start 
program, after-school and summer activities for children and 
youth. Referrals to local schools and agencies for such services as 
postsecondary education, driver’s education. Services for foreign- 
born residents (such as Hmong Women's Support Group for men- 
tal health and cultural conflict issues) provided by Hmong- and 
Spanish-speaking Jobs-Plus staff and partnerships with refugee 
organizations. Collocated employment counselor, financial eligibil- 
ity worker, and intensive case manager from welfare agency for 
TANF recipients. 


Community center provides employment assistance through St. 
Paul Public School's Support for Training and Employment Pro- 
gram (including job readiness, job search, computer classes, and 
skills training) as well as GED and ESL classes. Head Start and 
after-school and summer activities for children and youth, and full- 
service medical clinic. 


Rent Incentives 


First Jobs-Plus site to implement rent incentives, in November 
1998. Struggled with unexpected delays in HUD’s agreeing to 
cover potential losses to housing authority’s rent revenues incurred 
by permitting working households to keep more earnings. St. 
Paul’s plan provides one month's free rent for enrolling in Jobs- 
Plus. During Year 1, 100 percent of a household’s earned income 
is disregarded in calculation of monthly rent. In Years 2 through 5, 
rent calculations are based on the flat-rent model and are graduated 
to reflect a percentage (after utility adjustments) of the housing 
authority’s ceiling rents, ranging from 45 percent in Year 2 to 90 
percent in Year 5. Annual free month's rent if employed 12 con- 
tinuous months, plus $25 per month of deferred rent credit for each 
month of employment. Rent reductions during periods of unem- 
ployment. Strong management office support in recruiting, orient- 
ing, and enrolling households as well as administering incentives. 


Began implementing QHWRA rent incentives in 1999. 


Community 
Support for Work 


In 2000, residents who speak the languages of the most numerous 
ethnic groups at Mt. Airy Homes, under the direction of a Vista 
worker, recruited to help Jobs-Plus as community outreach workers 
to publicize program activities and job opportunities, recruit par- 
ticipants, and relay resident concerns to the staff. 


No formal efforts to engage residents in promoting employment 
services and job opportunities to other residents. 
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Appendix Table B.6 
Seattle: 

Comparison of Local Context, Services, and Incentives at the Jobs-Plus Development and Its Comparison Development 



Program Feature 


Jobs-Plus Development 

Rainier Vista Garden Community 


Comparison Development 

Yesler Terrace 


Housing and 
Neighborhood 


Development includes 481 one-story duplex apartment units in 
Rainier Valley, 5 miles southeast of Seattle’s city center. Sur- 
rounded by recent commercial and retail development, near large 
public park and expensive waterfront property. In 1999, awarded 
HOPE VI redevelopment funding, and temporary relocation of 
residents began in 2001 (although some residents only moved to 
other side of development). 


Development includes almost 700 apartment units in 93 wooden 
frame buildings, with community center. On steep hill, across 
from Seattle University, near Seattle's downtown and Interna- 
tional District, Harborview medical complex, Seattle Central 
Community College, which are sources of social and health ser- 
vices and jobs for Yesler Terrace residents. 


Demographics 


Ethnically diverse with African- and Caucasian-American resi- 
dents living alongside immigrants from East Africa, Southeast 
Asia, and Latin America. Majority female-headed households. 
Language- and immigrant-related barriers to outreach, service 
delivery, and employment, such as limited English proficiency 
and post-traumatic stress disorder. 


Similar to Rainier Vista demographic profile, but with more senior 
and disabled residents. 


Employment- 
Related Services 


Jobs-Plus at Rainier Vista became HOPE-Plus, offering employ- 
ment and support services and community-building assistance as 
part of the HOPE VI community and supportive services compo- 
nent. Once enrolled in HOPE-Plus, residents undergo intake and 
assessment with job coach. Participants with limited English pro- 
ficiency work with job coach who is fluent in their language, and 
can enroll in ESL classes offered by Refugee Women's Alliance, a 
collaborating organization that serves immigrants. Residents have 
access to help with resume preparation, job search, job retention, 
and career advancement, and instruction about workplace expecta- 
tions and comportment. 


Employment services for Seattle Housing Authority residents with 
children and on welfare at least 30 months through Job Connec- 
tion, U.S. Department of Labor-funded welfare-to-work program. 
If ineligible for Job Connection, residents referred to the Seattle 
Jobs Initiative’s local for-profit contractor, TRAC Associates. 
TRAC office mile away, paid on success in placing clients into 
jobs paying $8 or more per hour. Support service agencies on-site 
or nearby through Neighborhood House, Horn of Africa Services, 
Asian Counseling and Referral Services, and 
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Employment- 
Related Services 
(continued) 


Support services for work are wide-ranging, including help with 
child care and transportation and filing immigration-related pa- 
pers, Workshops available for filing income tax returns, business 
start-up, homeownership, financial management, and household 
budgeting skills. The job developer works with local employers to 
find suitable positions for residents. 


International District Housing Alliance. On-site Head Start center 
and youth programs and computer learning center supported by 
Seattle Parks and Recreation Department. Until 2001, Seattle- 
King County Public Health Department operated on-site health 
clinic. New community recreation center scheduled for fall 2004 
opening. In general, much less intensive on-site assistance than at 
Rainier Vista. 


Rent Incentives 


Implemented in September 1999, Seattle’s plan is based on series of 
steps in which rent levels gradually increased to market rate. In Step 
1, rents of working households were frozen at current level. In Steps 
2 through 4, rent increased every two years until equaling 100 per- 
cent of market rate for given apartment size (for instance. Step 2, 40 
percent of market rate; Step 3, 75 percent of market rate). Housing 
authority diverted portion of monthly rent payment to interest- 
bearing escrow account that resident could access to pay for educa- 
tion, business start-up, or homeownership. Any Rainier Vista adult 
resident in good standing was eligible to enroll until March 2001, 
including elderly and disabled, as well as those who were already 
working. Seattle Housing Authority assigned senior staff person on- 
site at Rainier Vista to administer the program. 


In 2000, implemented authority-wide rent incentives program 
under federal Moving to Work demonstration. Uses a system of 
gradually increasing ceiling rents. The ceiling establishes maxi- 
mum rent for successive two-year periods. First two years of em- 
ployment, rent ceiling $260 per month. Second two years, rent 
ceiling is $390. Third and final rent ceiling sets household rent at 
100 percent of fair market value of unit. Any time working resi- 
dents paying at least $390 in monthly rent, portion of difference 
between $350 and rent actually paying is contributed to savings 
account by housing authority. Savings available to household 
upon moving from public housing. 


Community 
Support for Work 


Residents who spoke languages of ethnic groups at Rainier Vista 
hired and trained as resident outreach and orientation specialists 
to publicize Jobs-Plus services and job opportunities and recruit 
participants. Also, Community Shares program permitted resi- 
dents to contribute services (for instance, attending meetings, serv- 
ing on committees, providing transportation) in return for credits 
that could be exchanged for $50 rent reduction or material re- 
sources, such as a computer. 


No formal efforts to engage residents in promoting employment 
services and job opportunities to other residents. 
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Appendix Table B.7 
Move-Out Rates for the 1998 Cohort 







Percentage Moving Out Within: 




Site and Development 


One Year 


Two Years 


Three Years 


Four Years 


Baltimore 


Jobs-Plus development 


30 


41 


51 


65 


Comparison developments 


21 


29 


35 


46 


Chattanooga 


Jobs-Plus development 


26 


36 


52 


63 


Comparison developments 


27 


40 


59 


71 


Davton 


Jobs-Plus development 


35 


48 


58 


67 


Comparison developments 


44 


54 


67 


72 


Los Angeles 


Jobs-Plus development 


9 


17 


22 


27 


Comparison development 


15 


24 


30 


46 


St. Paul 


Jobs-Plus development 


14 


27 


38 


51 


Comparison development 


31 


39 


49 


56 


Seattle 


Jobs-Plus development 


11 


18 


31 


52 


Comparison development 


16 


28 


37 


43 


All sites combined 


Jobs-Plus developments 


21 


31 


42 


54 


Comparison developments 


26 


36 


46 


56 



SOURCE: MDRC calculations using data from housing authority tenant (50058) records. 



NOTES: The 1998 cohort includes all residents of a Jobs-Plus development or a comparison development in October 
1998 who were between 21 and 61 years old and were not listed as disabled by their public housing authority. 

The follow-up period for calculating the move-out rates began in October 1998. 
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Appendix Table B.8 



Rates of Participation in Various Jobs-Plus Activities 
Among Jobs-Plus Enrollees Who Lived in the Developments Between 1998 and 2000 



Participation Measure (%) 


Baltimore 


Chattanooga 


Dayton 


Los Angeles 


St. Paul 


Seattle 


All 

Developments 

Combined 


Of those enrolled 11 


Ever oriented or assessed 


72 


99 


94 


82 


100 


NA 


NA 


Ever referred to a job 


41 


24 


65 


60 


43 


NA 


NA 


Ever referred to or participated in: 


Job club/search 


40 


30 


30 


30 


19 


19 


28 


English as a Second Language (ESL) course 


0 


0 


0 


23 


12 


5 


7 


Adult Basic Education (ABE) 


5 


18 


16 


4 


16 


1 


10 


Postsecondary education 


0 


1 


3 


9 


4 


0 


3 


Vocational training 


5 


9 


5 


18 


32 


3 


12 


Work experience 


0 


1 


3 


26 


3 


6 


7 


Life-skills training 


21 


16 


8 


13 


49 


4 


18 


Any of the above activities' 3 


48 


47 


46 


70 


83 


31 


54 


Any of the above activities 


or referred to a job 


60 


49 


76 


80 


83 


NA 


70 


Sample size 


58 


98 


63 


39 


134 


256 


648 
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SOURCES: MDRC calculations using data from Jobs-Plus case files and housing authority tenant (50058) records. 

NOTES: The target sample includes all nondisabled residents aged 18 to 61 living in a household headed by a nondisabled resident between ages 18 and 
61. 

Participation measures were adjusted based on the findings of an effort undertaken to confirm the accuracy of data collected from the Jobs-Plus case 
files. This effort involved randomly selecting approximately 20 enrollees from each Jobs-Plus development. Jobs-Plus staff reviewed the collected data to 
see if any information had been missed about enrollees’ service use. 

In the averages for all developments combined, the results for each housing development are weighted equally. 

J Basic information about the characteristics of Jobs-Plus enrollees at the time of enrollment was collected for all Jobs-Plus enrollees. This table 
includes additional information on participation in Jobs-Plus activities that was collected for a random subsample of enrollees. 
b Does not include orientation or assessment. 
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Appendix Table B.9 

2003 Follow-Up Survey (Household Heads) 

Use of Services and Incentives by Residents of the Jobs-Plus Developments and 
Their Comparison Developments, by Site (Excluding Seattle) 

Baltimore 



Measure (%) 


Jobs-Plus 

Development 


Comparison 

Developments 


Difference 


Participation and service use 

Participated in any employment-related activity 
within prior 12 months with help from: 

Any program/agency 


52.0 


40.1 


1 j 9 *** 


Housing authority or any program at development 


29.1 


13.2 


[5 9 *** 


Within prior 12 months, participated in: 
Any job search activity 


41.1 


29.6 


11.5 *** 


Any education or training activity 


34.6 


32.9 


1.7 


Received any ancillary or social services 
within prior 12 months with help from: 
Any program/agency 


32.0 


33.7 


-1.7 


Housing authority or any program at development 


16.8 


8.9 


7 9 *** 


Financial incentives 

Heard of rent-based work incentives 


46.9 


20.4 


26.4 *** 


Currently using rent-based work incentives 


20.1 


11.9 


8.2 ** 


Currently using rent-based work incentives (among 
currently working residents only) a 


28.5 


16.2 


NA 


Heard of EITC b 


69.8 


53.2 


16.6 *** 


Was encouraged to use EITC by housing 
authority or any program at development 


17.7 


7.0 


j0 7 *** 


Used EITC during prior year 


24.2 


19.6 


4.6 


Services and incentives 

Participated in any employment-related services within prior 
12 months, or currently using rent-based work 
incentives 


41.0 


30.6 


10.4 ** 


Overall current self-sufficiencv efforts 

Currently participating in any work-related activity, currently 
working, or looked for work within the past four weeks 


69.9 


64.1 


5.8 


Sample size 


203 


303 
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Appendix Table B.9 (continued) 



Chattanooga 





Jobs-Plus 


Comparison 




Measure (%) 


Development 


Developments 


Difference 


Participation and service use 








Participated in any employment-related activity 
within prior 12 months with help from: 








Any program/agency 


65.2 


39.7 


25.5 *** 


Housing authority or any program at development 


29.4 


10.5 


18.8 *** 


Within prior 12 months, participated in: 








Any job search activity 


48.1 


23.7 


24.4 *** 


Any education or training activity 


45.2 


25.5 


19 7 *** 


Received any ancillary or social services 
within prior 12 months with help from: 








Any program/agency 


57.0 


47.0 


10.0 


Housing authority or any program at development 


25.0 


12.6 


12.4 ** 


Financial incentives 








Heard of rent-based work incentives 


74.4 


55.9 


18.5 ** 


Currently using rent-based work incentives 


49.8 


21.9 


27 9 *** 


Currently using rent-based work incentives (among 
currently working residents only) 11 


53.3 


28.3 


NA 


Heard of EITC D 


58.8 


52.5 


6.3 


Was encouraged to use EITC by housing 
authority or any program at development 


8.1 


2.6 


5.5 


Used EITC during prior year 


30.0 


20.7 


9.3 


Services and incentives 








Participated in any employment-related activities within prior 

12 months or currently using rent-based work 

incentives 


65.8 


40.3 


25.5 *** 


Overall current self-sufficiencv efforts 








Currently participating in any work-related activity, currently 
working, or looked for work within the past four weeks 


81.7 


69.5 


12.2 * 


Sample size 


86 


83 
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Appendix Table B.9 (continued) 



Dayton 



Measure (%) 


Jobs-Plus 

Development 


Comparison 

Developments 


Difference 


Participation and service use 

Participated in any employment-related activity 
within prior 12 months with help from: 

Any program/agency 


58.6 


53.1 


5.5 


Housing authority or any program at development 


30.8 


8.8 


2 i 9 *** 


Within prior 12 months, participated in: 
Any job search activity 


47.0 


39.6 


7.4 


Any education or training activity 


49.1 


44.4 


4.7 


Received any ancillary or social services 
within prior 12 months with help from: 
Any program/agency 


43.9 


38.0 


5.9 


Housing authority or any program at development 


25.2 


7.0 


18.2 *** 


Financial incentives 

Heard of rent-based work incentives 


64.1 


41.2 


22 9 *** 


Currently using rent-based work incentives 


49.7 


26.6 


23 i *** 


Currently using rent-based work incentives (among 
currently working residents only) a 


56.0 


34.9 


NA 


Heard of EITC b 


70.9 


69.3 


1.6 


Was encouraged to use EITC by housing 
authority or any program at development 


12.0 


3.8 


g 2 *** 


Used EITC during prior year 


35.4 


29.0 


6.3 


Services and incentives 

Participated in any employment-related activities within prior 

12 months or currently using rent-based work 

incentives 


64.0 


46.6 


17 4 *** 


Overall current self-sufficiencv efforts 

Currently participating in any work-related activity, currently 
working, or looked for work within the past four weeks 


77.2 


66.3 


no** 


Sample size 


181 


275 
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Appendix Table B.9 (continued) 



Los Angeles 



Measure (%) 


Jobs-Plus 

Development 


Comparison 

Development 


Difference 


Participation and service use 

Participated in any employment-related activity 
within prior 12 months with help from: 

Any program/agency 


48.1 


39.5 


8.5 


Housing authority or any program at development 


23.4 


7.4 


16.0 ** 


Within prior 12 months, participated in: 
Any job search activity 


19.3 


1.0 


18.3 *** 


Any education or training activity 


42.6 


38.9 


3.8 


Received any ancillary or social services 
within prior 12 months with help from: 
Any program/agency 


51.4 


45.2 


6.2 


Housing authority or any program at development 


32.5 


16.7 


15.9 * 


Financial incentives 

Heard of rent-based work incentives 


52.5 


35.6 


16.9 * 


Currently using rent-based work incentives 


47.9 


30.8 


17.1 * 


Currently using rent-based work incentives (among 
currently working residents only)" 


51.2 


35.2 


NA 


Heard of EITC b 


24.8 


21.3 


3.4 


Was encouraged to use EITC by housing 
authority or any program at development 


10.4 


7.5 


2.9 


Used EITC during prior year 


5.6 


7.6 


-2.0 


Services and incentives 

Participated in any employment-related activities within prior 

12 months or currently using rent-based work 

incentives 


64.0 


51.1 


13.0 


Overall current self-sufficiencv efforts 

Currently participating in any work-related activity, currently 
working, or looked for work within the past four weeks 


89.4 


87.5 


1.8 


Sample size 


85 


56 
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Appendix Table B.9 (continued) 
St. Paul 



Measure (%) 


Jobs-Plus 

Development 


Comparison 

Development 


Difference 


Participation and service use 

Participated in any employment-related activity 
within prior 12 months with help from: 

Any program/agency 


59.6 


52.4 


7.2 


Housing authority or any program at development 


40.2 


25.9 


14.3 ** 


Within prior 12 months, participated in: 
Any job search activity 


41.2 


32.3 


8.9 


Any education or training activity 


45.2 


47.1 


-1.9 


Received any ancillary or social services 
within prior 12 months with help from: 
Any program/agency 


70.0 


49.3 


20 7 *** 


Housing authority or any program at development 


48.3 


8.2 


40.1 *** 


Financial incentives 

Heard of rent-based work incentives 


78.7 


49.5 


29 2 *** 


Currently using rent-based work incentives 


55.3 


28.5 


26.8 *** 


Currently using rent-based work incentives (among 
currently working residents only) a 


72.8 


42.2 


NA 


Heard of EITC b 


NA 


NA 


NA 


Was encouraged to use EITC by housing 
authority or any program at development 


14.5 


10.5 


4.0 


Used EITC during prior year 


66.1 


54.1 


12.0 


Services and incentives 

Participated in any employment-related activities within prior 

12 months or currently using rent-based work 

incentives 


70.6 


56.3 


14.3 ** 


Overall current self-sufficiencv efforts 

Currently participating in any work-related activity, currently 
working, or looked for work within the past four weeks 


90.2 


83.6 


6.5 


Sample size 


113 


91 
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Appendix Table B.9 (continued) 

SOURCE: MDRC calculations using data from the 2003 follow-up survey. 

NOTES: Estimates of program-comparison development differences control for various background 
characteristics of respondents. 

Statistical significance levels are indicated as *** = 1 percent; ** = 5 percent; and * = 10 percent. 
“Because the calculations for this measure exclude respondents who were not working at the time of the 
interview, rather than all respondents, no differences in take-up rates between the Jobs-Plus and comparison 
developments are computed. 

Respondents in St. Paul were not asked this survey question. 
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Appendix Table B.10 

2003 Follow-Up Survey (Household Heads) 

Use of Services and Incentives by Residents of the Jobs-Plus Developments and 
Their Comparison Developments in the Three Stronger Implementation Sites 

Combined: Welfare Subsample 



Measure (%) 


Jobs-Plus 

Developments 


Comparison 

Developments 


Difference 


Participation and service use 

Participated in any employment-related activity 
within prior 1 2 months with help from: 

Any program/agency 


72.0 


64.3 


7.7 


Housing authority or any program at development 


42.1 


17.2 


25.0 *** 


Within prior 12 months, participated in: 
Any job search activity 


54.5 


41.3 


13.2 ** 


Any education or training activity 


56.1 


53.8 


2.3 


Received any ancillary or social services 
within prior 1 2 months with help from: 
Any program/agency 


70.2 


49.3 


20 9 *** 


Housing authority or any program at development 


49.4 


10.1 


39 3 *** 


Financial incentives 

Heard of rent-based work incentives 


66.2 


40.9 


25.3 *** 


Currently using rent-based work incentives 


46.9 


19.5 


27 4 *** 


Currently using rent-based work incentives (among 
currently working residents only) a 


60.4 


33.0 


NA 


Heard ofEITC b 


70.8 


60.1 


10.7 


Was encouraged to use EITC by housing 
authority or any program at development 


14.5 


5.7 


8.8 ** 


Used EITC during prior year 


42.7 


31.6 


11.2 * 


Services and incentives 

Participated in any employment-related activities within prior 

12 months or currently using rent-based work 

incentives 


72.0 


47.3 


24 7 *** 


Overall current self-sufficiencv efforts 

Currently participating in any work-related activity, currently 
working, or looked for work within the past four weeks 


83.2 


71.7 


11.5 ** 


Sample size 


133 


150 





(continued) 
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Appendix Table B.10 (continued) 



SOURCE: MDRC calculations using data from the 2003 follow-up survey. 

NOTES: Estimates of program-comparison development differences control for various background 
characteristics of respondents. 

Statistical significance levels are indicated as *** = 1 percent; ** = 5 percent; and * = 10 percent. 
“Because the calculations for this measure exclude respondents who were not working at the time of the 
interview, rather than all respondents, no differences in take-up rates between the Jobs-Plus and comparison 
developments are computed. 

b Respondents in St. Paul were not asked this survey question. 
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The Jobs-Plus Demonstration 



Appendix Table B.ll 

2003 Follow-Up Survey (Household Heads) 

Use of Services and Incentives by Residents of the Jobs-Plus Developments and 
Their Comparison Developments in the Three Stronger Implementation Sites 
Combined: Nonwelfare Subsample 



Measure (%) 


Jobs-Plus Comparison 
Developments Developments 


Difference 


Participation and service use 

Participated in any employment-related activity 
within prior 12 months with help from: 

Any program/agency 


49.2 


42.1 


7.1 


Housing authority or any program at development 


27.3 


12.0 


15.3 *** 


Within prior 12 months, participated in: 
Any job search activity 


32.3 


22.3 


10.0 ** 


Any education or training activity 


41.6 


39.4 


2.1 


Received any ancillary or social services 
within prior 12 months with help from: 
Any program/agency 


47.0 


42.2 


4.8 


Housing authority or any program at development 


27.8 


10.6 


17 2 *** 


Financial incentives 

Heard of rent-based work incentives 


70.7 


44.9 


25.8 *** 


Currently using rent-based work incentives 


57.0 


34.5 


22.5 *** 


Currently using rent-based work incentives ( among 
currently working residents only) a 


64.5 


40.6 


NA 


Heard of EITC b 


53.6 


55.4 


-1.8 


Was encouraged to use EITC by housing 
authority or any program at development 


10.0 


7.0 


2.9 


Used EITC during prior year 


39.0 


33.3 


5.7 


Services and incentives 

Participated in any employment-related activities within prior 

12 months or currently using rent-based work 

incentives 


65.8 


53.0 


12.8 *** 


Overall current self-sufficiencv efforts 

Currently participating in any work-related activity, currently 
working, or looked for work within the past four weeks 


85.1 


80.0 


5.2 


Sample size 


246 


272 





(continued) 
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Appendix Table B.ll (continued) 



SOURCE: MDRC calculations using data from the 2003 follow-up survey. 

NOTES: Estimates of program-comparison development differences control for various background 
characteristics of respondents. 

Statistical significance levels are indicated as *** = 1 percent; ** = 5 percent; and * = 10 percent. 
“Because the calculations for this measure exclude respondents who were not working at the time of the 
interview, rather than all respondents, no differences in take-up rates between the Jobs-Plus and comparison 
developments are computed. 

b Respondents in St. Paul were not asked this survey question. 
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Appendix C 

Baseline and Follow-Up Surveys 
of Residents 




This appendix describes the two surveys of residents that were conducted for the Jobs- 
Plus evaluation: a baseline survey , fielded near the start of Jobs-Plus, and a. follow-up survey , 
fielded during the final year of its operation. Chapters 2, 3, and 6 draw on findings from one or 
both of these surveys to describe the participation of residents in Jobs-Plus and other employment- 
related activities and to describe how residents’ living conditions and the conditions in their hous- 
ing developments changed over time. The sections that follow describe how samples were se- 
lected for the two surveys, present the survey response rates, and compare the background charac- 
teristics of respondents from the Jobs-Plus developments and from the comparison developments. 



The Surveys 

The baseline and follow-up surveys for Jobs-Plus provide two waves of repeated cross- 
sectional data on people living in the Jobs-Plus developments and their comparison develop- 
ments. 1 The pre- Jobs-Plus picture is captured by the baseline survey, which was a 45-minute 
interview with a sample of eligible household heads. In five of the six sites — Baltimore, Day- 
ton, Chattanooga, Los Angeles, and Seattle — the baseline survey was conducted in the spring 
and summer of 1998. In St. Paul, the survey was conducted in 1999. 2 

The follow-up survey of eligible household heads from each housing development was 
conducted during the summer of 2003, roughly five years into the demonstration. A full follow- 
up survey was conducted in three sites — Baltimore, Dayton, and St. Paul — using the same 
procedures that were used to administer the baseline survey. Due to cost considerations, a re- 
duced version of the survey — or “mini-survey” — was administered to smaller samples in 
Chattanooga and Los Angeles. No follow-up survey was conducted in Seattle because that site 
was withdrawn from the Jobs-Plus demonstration when it launched a HOPE VI renovation in 
the Jobs-Plus development. 

Comprehensive baseline and follow-up survey data are therefore available for Balti- 
more, Dayton, and St. Paul. As a result, the community change analysis in Chapter 6 focuses on 
these three sites. The mini-survey, administered in Los Angeles and Chattanooga, focused only 
on respondents’ participation in Jobs-Plus or in other employment-related activities plus their 
receipt of ancillary services, such as transportation payments, child care assistance, and pay- 
ments for books, tools, or uniforms (which was also examined by the full follow-up survey). 
This information is used in Chapter 3 to examine the services received by residents of the Jobs- 
Plus and comparison developments. 



'Residents who were interviewed for the baseline survey and then moved out of their developments were 
not followed up. 

2 It took longer to develop and launch the baseline survey in St. Paul, given the need to select a specialized 
survey contractor and to develop a survey that could be administered in Hmong as well as in English. 
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Each survey attempted to interview a sample of working-age, nondisabled household 
heads who were identified from the records of their local public housing authority. Where the 
household head was not available or could not speak any language in which the survey was con- 
ducted, another adult in the household was interviewed, if possible. Interviews were conducted 
mainly by phone using a Computer Assisted Telephone Interviewing (CATI) system . 3 For sample 
members who could not be reached by phone, the survey contractor used trackers with cell phones 
to enable sample members to call the survey telephone center in order to be interviewed. 

The surveys were administered in English, Spanish, and Hmong . 4 They were first de- 
veloped in English and then were translated into the other two languages. Care was taken to en- 
sure the accuracy and precision of the translations. 

The surveys covered a wide variety of topics to gauge and explain sites’ success or fail- 
ure in implementing Jobs-Plus and to assess the effects of the program on a range of individual 
and community outcomes. Survey modules are listed below, with a brief description of their 
content. The baseline survey and the full follow-up survey contained all the modules listed. The 
mini-survey focused mainly on the first module. 

• Education, training, program participation, and program experiences. 

This module of the baseline survey focused on residents' readiness for work 
when Jobs-Plus was being launched. It asked questions about education and 
job-related training received before Jobs-Plus began. This module of the fol- 
low-up survey asked questions about residents’ knowledge and/or use of the 
opportunities provided by Jobs-Plus or other local programs in three core ar- 
eas: employment and training activities and related support services, financial 
incentives to work, and community support for work. 

• Work and employment. This module focused on residents’ current or recent 
employment, job characteristics, benefits available through work, and job- 
seeking behaviors. It also asked about residents' preferences for work and their 
access to information about work both inside and outside the community. 

• Material well-being and income. This module asked about residents’ eco- 
nomic and material circumstances. Questions examined the sources of resi- 
dents’ household income, the total amount of this income, the material 
hardships encountered by residents, and their satisfaction with their stan- 
dard of living. 



3 In St. Paul, given the large number of non-English-speaking sample members, the survey contractor for 
the site used paper-and-pencil interviews (PAPI). 

4 The Battelle Centers for Public Health Research and Evaluation conducted the baseline and follow-up 
surveys at all sites except St. Paul, where the surveys were conducted by the Wilder Research Center. 
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• Health and well-being. This module asked about mental and physical health 
issues that might affect residents’ abilities to obtain and retain work. It also 
asked about residents' medical insurance coverage, interpersonal difficulties, 
perceived neighborhood safety, and past criminal convictions. 

• Community life. This module asked about residents’ perceptions of life and 
conditions in their housing developments. It attempted to document the social 
context that might affect — and be affected by — the implementation of 
Jobs-Plus. The module asked residents about how long they had lived in their 
development, how they would rate it as a place to live, their perceptions of 
problem conditions in the development, and the degree of social cohesion 
that existed among their neighbors. The module also asked about participa- 
tion in neighborhood activities and attachment to various social networks. 

• Children. This module collected data on children who lived in respondents’ 
households. Some questions were asked about young children ages 6 to 11, 
and other questions were asked about older children ages 12 to 17. Questions 
examined children’s school and nonschool activities; their rewards and rec- 
ognitions; and behavioral problems that they experienced, such as school 
suspensions or expulsions. The St. Paul survey included very few questions 
in this module because of the inability to translate many of the ideas into 
concepts that were meaningful to Hmong respondents. 

• Background characteristics. This module of both surveys asked about resi- 
dents’ demographic characteristics, country of origin, public housing history, 
and legal status with respect to employment in the United States. 



Sample Sizes 

The baseline and follow-up surveys in all sites were administered to randomly selected 
heads of households who were (1) living in a Jobs-Plus or comparison development at the time 
of sample selection and survey administration, 5 (2) not identified by the public housing authority 
as being disabled, and (3) between ages 21 and 61 when they were sampled. Hence, respon- 
dents were persons who were eligible to participate in Jobs-Plus and who were capable of re- 
porting on the quality of life in their housing development. 



5 Sample members who were interviewed for the baseline survey had to be living in the development at the 
point of sample selection (October 1997) and six months later, when the interviews began. To be eligible for 
the follow-up survey, people had to have lived in the development for at least a year. In other words, to be in 
the eligible sample, they had to show up on the October 2001 and October 2002 housing authority records. Of 
course, they also had to be living in the development at the time of the interview. 



204 




The October 1997 housing authority records were used to identify the baseline survey 
sample. Applying the above selection criteria to these records yielded a sampling frame of 3,841 
eligible persons. For each site, a random sample of approximately 330 persons was selected 
from the Jobs-Plus development, and a random sample of approximately 330 persons was se- 
lected from the comparison development/s. For sites with two comparison developments, the 
comparison sample was split evenly between them. 6 

The October 2002 housing authority records were used to identify the follow-up survey 
sample. These records yielded a sampling frame of 2,435 eligible persons. A random sample of 
approximately 685 persons was selected from the Jobs-Plus developments, and 828 were se- 
lected from the comparison developments. 

Appendix Table C.l presents the sample sizes for the Jobs-Plus group and the compari- 
son group in the baseline and follow-up surveys, by site. The first two columns in the table list 
the sample size for each survey. The third column lists the number of persons who were inter- 
viewed for both surveys (the “overlapping subsample”)- And the rightmost column reports the 
size of the overlapping subsample as a percentage of the size of the baseline sample. 

The table reflects two main findings. First it indicates that the overlap between the base- 
line sample and the follow-up sample is quite small in all sites. Thus, findings from the two sur- 
veys reflect not only changes in conditions that occurred within developments over time but 
also changes in the persons who reported on these conditions. Second, the degree of overlap 
between the baseline and follow-up samples varies substantially across sites and developments: 
from a low of 5.3 percent for the comparison developments in Chattanooga to a high of 41.7 
percent for the Jobs-Plus development in St. Paul. This variation in overlap reflects (imper- 
fectly) the variation in resident mobility across sites. 7 



Response Rates 

Appendix Table C.2 presents survey response rates for the Jobs-Plus and comparison 
developments from each site. The first column in the table presents response rates for the base- 
line survey. The second column presents response rates for the full follow-up survey. And the 
rightmost column presents response rates for the mini follow-up survey. 

Response rates for the baseline survey are substantial overall, averaging 87.5 percent 
for the Jobs-Plus developments and 79.6 percent for the comparison developments. Thus, in 
general, there is not much room for response bias to create artificial differences in results for the 
Jobs-Plus developments and comparison developments. However, there is appreciable variation 



6 At developments with fewer than 330 eligible respondents, all of them were included in the sample. 

7 As Table 3.2 indicates, move-out rates were highest in Baltimore, Chattanooga, and Dayton and consid- 
erably lower in Los Angeles and St. Paul. 
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The Jobs-Plus Demonstration 

Appendix Table C.l 



Baseline and Follow-Up Survey Sample Sizes, by Site 





Sample Size 




Respondents to Both Surveys 


Site and Development 


Baseline Survey Follow-Up Survey 


Number 


Percentage 3 


Baltimore 


Jobs-Plus development 


219 


219 


57 


26.0 


Comparison developments 


215 


323 


70 


32.6 


Chattanooaa 1 ’ 


Jobs-Plus development 


220 


87 


23 


10.5 


Comparison developments 


225 


83 


12 


5.3 


Davton 


Jobs-Plus development 


241 


181 


48 


19.9 


Comparison developments 


287 


275 


51 


17.8 


Los An2eles b 


Jobs-Plus development 


237 


85 


33 


13.9 


Comparison development 


179 


56 


17 


9.5 


St. Paul 


Jobs-Plus development 


151 


113 


63 


41.7 


Comparison development 


119 


91 


42 


35.3 


Totals for Jobs-Plus developments 


1,068 


685 


224 


22.4 


Totals for comparison developments 




828 


192 


20.1 



SOURCES: MDRC calculations using data from the baseline and follow-up surveys. 

NOTES: No follow-up survey was conducted in Seattle because that site was withdrawn from the Jobs-Plus 
demonstration when it launched a HOPE VI renovation in the Jobs-Plus development. 

a The number of respondents to both surveys as a percentage of the number of baseline respondents. 
b Due to resource limitations, a much briefer follow-up survey was administered to a smaller sample of 
residents in Chattanooga and Los Angeles. 



in response rates across sites — ranging from a high of 91.6 percent for the Jobs-Plus develop- 
ment in Dayton to a low of 67.8 percent for the comparison development in Los Angeles. Fur- 
thermore, the response rates for both the Jobs-Plus development and the comparison develop- 
ment in Los Angeles are noticeably lower than the rates for their counterparts in other sites. 
Hence, the margin for response bias in the baseline survey is greatest for Los Angeles. 
Lastly, note that the response rate for the Jobs-Plus sample is somewhat greater than the rate 
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The Jobs-Plus Demonstration 
Appendix Table C.2 



Jobs-Plus Baseline and Follow-Up Survey Response Rates, by Development, 

for the Eligible Fielded Sample 



Site and Development 


Baseline 

Survey 


Full Follow-Up 
Survey 


Mini Follow-Up 
Survey 


Baltimore 


Jobs-Plus development 


86.6 


89.4 


NA 


Comparison developments 


76.8 


80.5 


NA 


Chattanooga 3 


Jobs-Plus development 


89.4 


NA 


54.7 


Comparison developments 


75.5 


NA 


52.9 


Davton 


Jobs-Plus development 


91.6 


86.6 


NA 


Comparison developments 


87.8 


94.2 


NA 


Los Angeles 3 


Jobs-Plus development 


79.0 


NA 


77.3 


Comparison development 


67.8 


NA 


57.1 


St. Paul 


Jobs-Plus development 


91.0 


95.8 


NA 


Comparison development 


90.2 


88.3 


NA 


Total response rate for Jobs-Plus developments 


87.5 


90.6 




Total response rate for comparison developments 


79.6 


87.7 





SOURCES: MDRC calculations using data from the baseline and follow-up surveys. 

NOTES: No follow-up survey was conducted in Seattle because that site was withdrawn from the Jobs- 
Plus demonstration when it launched a HOPE VI renovation in the Jobs-Plus development. 

■“Due to resource limitations, a much briefer follow-up survey was administered to residents in 
Chattanooga and Los Angeles. 



for the comparison sample at all sites. In sites where response rates are high overall, this is not 
likely to be a problem. But it could be problematic in Los Angeles, and so caution should be 
used when interpreting survey results for this site. 

Response rates for the full follow-up survey in Baltimore, Dayton, and St. Paul are very 
high overall, averaging 90.6 percent for the Jobs-Plus developments and 87.7 percent for the 
comparison developments. Furthermore, they are very high at all sites, and there is no consistent 
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pattern between response rates for the Jobs-Plus sample and for the comparison sample. Hence, 
there is little margin for response bias to influence the findings of this survey. 

Response rates for the mini follow-up survey are relatively low, averaging 53.8 percent 
in Chattanooga and 67.2 percent in Los Angeles. In addition, they are generally low for all de- 
velopments at both sites. Thus, there is a substantial margin for response bias to influence re- 
sults from this survey. On the other hand, there is little difference (and no consistent pattern in 
the differences) between response rates for the Jobs-Plus samples and the comparison samples. 
Thus, the response rates themselves do not provide evidence of differential response bias. Nev- 
ertheless, extra caution is in order when interpreting the follow-up survey results, particularly 
for Los Angeles and Chattanooga. 



The Comparability of Survey Respondents from the Jobs-Plus 
Developments and the Comparison Developments 

Table C.3 compares selected background characteristics of respondents to the baseline 
survey and respondents to the follow-up survey from both the Jobs-Plus development and the 
comparison development/s at each site. Ideally, there would be no differences among respon- 
dents to either survey, and thus — at least with respect to the characteristics that were observed 
and reported — the Jobs-Plus sample and the comparison sample for both surveys would be 
comparable. Any differences observed, however, must reflect a mix of two factors: (1) true dif- 
ferences in the average characteristics of eligible residents from the Jobs-Plus development and 
the comparison development/s (differences in their sampling frames) and (2) differences pro- 
duced by differential response bias. For the present analysis, the most relevant consideration is 
the extent to which the Jobs-Plus sample and the comparison sample are, in fact, alike, regard- 
less of what might cause differences between them. Thus, instead of separately examining the 
two potential sources of differences, Table C.3 focuses directly on the joint results, by site. 

The first two columns in Table C.3 describe the characteristics of baseline survey re- 
spondents from the Jobs-Plus development and the comparison development/s, respectively. 
The third column indicates whether the difference in each characteristic for the two groups of 
respondents is statistically significant. The three right-hand columns report corresponding in- 
formation for the follow-up survey respondents. The sizes of the survey samples are reported in 
the bottom row of each panel in the table. 

From a scan of the many results in the table, it appears that the survey samples for the 
Jobs-Plus developments are reasonably comparable to the samples for the comparison develop- 
ments. There are some isolated instances of sizable differences, and a number of differences are 
large enough to be statistically significant. On balance, however, from this simple set of compari- 
sons, there does not appear to be strong evidence of noncomparability between the survey respon- 
dents from the Jobs-Plus developments and those from the comparison developments. 
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The Jobs-Plus Demonstration 
Appendix Table C.3 

Differences Between Survey Respondents in the Jobs-Plus and Comparison Developments, 

by Selected Background Characteristics 

Baltimore 







Baseline Survey 






Follow-Up Survey 






Jobs-Plus 


Comparison 


Significance of 


Jobs-Plus 


Comparison 


Significance of 


Characteristic 


Respondents 


Respondents 


Difference 11 


Respondents 


Respondents 


Difference" 1 


Female (%) 


91.3 


83.3 


** 


89.6 


88.1 




Race/ethnicity (%) 






NA 






NA 


White 


0.0 


0.9 




0.0 


0.3 




Black 


99.5 


99.1 




99.1 


99.7 




Hispanic 

Asian 

Other 


0.5 


0.0 




0.9 


0.0 




Average age (in years) 


36.4 


39.1 


*** 


36.6 


39.0 


** 


Years in development (%) 






*** 






*** 


Less than 2 


2.3 


3.7 




7.3 


9.3 




2-5 


23.7 


11.2 




54.8 


39.3 




6-9 


47.9 


49.8 




18.7 


23.5 




10 or more 


26.0 


35.3 




19.2 


27.9 




Number of adults in household (%) 












** 


1 


89.5 


84.7 




95.7 


89.9 




2 or more 


10.5 


15.3 




4.3 


10.1 




Number of children in 
household (%) 












* 


0 


34.2 


40.9 




53.6 


56.8 




1 


26.5 


21.4 




26.6 


18.8 




2 or more 


39.3 


37.7 




19.8 


24.4 




Sample size 


219 


215 




219 


323 





(continued) 
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Appendix Table C.3 (continued) 
Chattanooga 







Baseline Survey 






Follow-Up Survey 






Jobs-Plus 


Comparison 


Significance of 


Jobs-Plus 


Comparison 


Significance of 


Characteristic 


Respondents 


Respondents 


Difference 11 


Respondents 


Respondents 


Difference 2 


Female (%) 


97.3 


96.4 




87.4 


90.4 




Race/ethnicity (%) 












NA 


White 


2.7 


2.2 




3.5 


1.2 




Black 


94.5 


95.5 




96.5 


98.8 




Hispanic 

Asian 

Other 


2.7 


2.2 










Average age (in years) 


30.5 


32.2 


* 


33.3 


36.5 


* 


Years in development (%) 












NA 


Less than 2 


2.7 


1.8 




0.0 


1.2 




2-5 


30.9 


31.6 




56.3 


62.7 




6-9 


41.8 


50.2 




20.7 


19.3 




10 or more 


24.5 


16.4 




23.0 


16.9 




Number of adults in household (%) 














1 


92.7 


95.1 




90.7 


94.0 




2 or more 


7.3 


4.9 




9.3 


6.0 




Number of children in 
household (%) 














0 


9.5 


15.1 




29.1 


44.6 




1 


26.8 


27.1 




26.7 


21.7 




2 or more 


63.6 


57.8 




44.2 


33.7 




Sample size 


220 


225 




87 


83 





(continued) 




Appendix Table C.3 (continued) 
Dayton 







Baseline Survey 






Follow-Up Survey 






Jobs-Plus 


Comparison 


Significance of 


Jobs-Plus 


Comparison 


Significance of 


Characteristic 


Respondents 


Respondents 


Difference 3 


Respondents 


Respondents 


Difference 11 


Female (%) 


92.1 


89.9 




90.1 


85.1 




Race/ethnicity (%) 






NA 






NA 


White 


0.4 


7.8 




0.0 


7.4 




Black 


99.2 


91.1 




100.0 


91.9 




Hispanic 


0.4 


0.4 




0.0 


0.4 




Asian 


0.0 


0.4 










Other 


0.0 


0.4 




0.0 


0.4 




Average age (in years) 


29.6 


30.0 




32.4 


32.2 




Years in development (%) 














Less than 2 


2.5 


4.2 




13.3 


19.3 




2-5 


44.4 


46.7 




49.2 


43.3 




6-9 


39.8 


34.8 




24.9 


24.0 




10 or more 


13.3 


14.3 




12.7 


13.5 




Number of adults in household (%) 






* 








1 


94.2 


89.5 




91.2 


88.0 




2 or more 


5.8 


10.5 




0° 

OO 


12.0 




Number of children in 
household (%) 












** 


0 


18.7 


20.6 




30.4 


22.9 




1 


33.6 


26.8 




30.9 


25.5 




2 or more 


47.7 


52.6 




38.7 


51.6 




Sample size 


241 


287 




181 


275 





(continued) 
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Appendix Table C.3 (continued) 



Los Angeles 







Baseline Survey 






Follow-Up Survey 






Jobs-Plus 


Comparison 


Significance of 


Jobs-Plus 


Comparison 


Significance of 


Characteristic 


Respondents 


Respondents 


Difference 3 


Respondents 


Respondents 


Difference 3 


Female (%) 


95.4 


91.6 




68.2 


57.1 




Race/ethnicity (%) 






NA 






NA 


White 


1.7 


2.4 




2.4 


0.0 




Black 


7.2 


11.2 




8.3 


12.5 




Hispanic 


81.4 


84.7 




86.9 


87.5 




Asian 


9.3 


1.8 




2.4 


0.0 




Other 


0.4 


0.0 










Average age (in years) 


40.2 


38.4 


* 


45.5 


41.5 


** 


Years in development (%) 






NA 








Less than 2 


0.4 


1.1 










2-5 


12.7 


15.6 




24.7 


32.1 




6-9 


46.4 


53.1 




30.6 


37.5 




10 or more 


40.5 


30.2 




44.7 


30.4 




Number of adults in household (%) 






** 








1 


45.6 


57.0 




45.9 


50.0 




2 or more 


54.4 


43.0 




54.1 


50.0 




Number of children in 
household (%) 






*** 






* 


0 


24.9 


22.9 




28.2 


12.5 




1 


27.8 


16.2 




23.5 


21.4 




2 or more 


47.3 


60.9 




48.2 


66.1 




Sample size 


237 


179 




85 


56 
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Appendix Table C.3 (continued) 
St. Paul 







Baseline Survey 






Follow-Up Survey 






Jobs-Plus 


Comparison 


Significance of 


Jobs-Plus 


Comparison 


Significance of 


Characteristic 


Respondents 


Respondents 


Difference 11 


Respondents 


Respondents 


Difference 11 


Female (%) 


70.2 


78.2 




73.5 


82.4 




Race/ethnicity (%) 






** 






** 


White 


2.4 


11.1 




4.2 


9.3 




Black 


29.8 


35.9 




38.5 


54.7 




Hispanic 


6.5 


3.4 




9.4 


4.7 




Asian 


60.5 


48.7 




46.9 


29.1 




Other 


0.8 


0.9 




1.0 


2.3 




Average age (in years) 


35.0 


31.8 


*** 


36.5 


32.1 


*** 


Years in development (%) 






NA 








Less than 2 


0.7 


4.2 










2-5 


29.8 


42.0 




49.6 


61.5 




6-9 


52.3 


41.2 




39.8 


27.5 




10 or more 


17.2 


12.6 




10.6 


11.0 




Number of adults in household (%) 






*** 






*** 


1 


47.7 


69.7 




52.2 


71.4 




2 or more 


52.3 


30.3 




47.8 


28.6 




Number of children in 
household (%) 






NA 






NA 


0 


1.3 


5.0 




1.8 


5.5 




1 


6.0 


31.9 




10.6 


26.4 




2 or more 


92.7 


63.0 




87.6 


68.1 




Sample size 


151 


119 




113 


91 
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Appendix Table C.3 (continued) 



SOURCES: MDRC calculations using baseline and follow-up surveys. 

NOTES: A chi-square test was applied to differences between the groups in distributions of characteristics. A two-tailed t-test was applied to differences 
between means for the groups. Statistical significance levels are indicated as *** = 1 percent; ** = 5 percent; and * = 10 percent. 

Missing values are not included in individual variable distributions. 

Distributions may not add to 100 percent because of rounding. 

■The significance of the difference between Jobs-Plus respondents and comparison respondents. 




Appendix D 

Estimating the Impacts of Jobs-Plus 
on Work and Welfare 




This appendix presents a step-by-step discussion of how the impacts of Jobs-Plus on 
work and welfare were estimated. These estimates were based on outcome data covering a six- 
year baseline period before Jobs-Plus was launched and a six-year follow-up period after the 
program was launched. 



The Analytical Perspectives and Measures 

A central feature of the evaluation of Jobs-Plus is its focus on program effectiveness 
from two perspectives. One perspective, which is referred to as the “individual perspective,” 
focuses on individual public housing residents (people). The other perspective, which is referred 
to as the “development perspective,” focuses on public housing developments (place). 

The individual perspective relates to a particular group of persons who were living in 
Jobs-Plus developments at a specific point in time. Thus, it examines the experiences over time 
of a given cohort of residents. From this perspective, the Jobs-Plus evaluation addresses the 
question “How did the program affect the future experiences of its target group members, re- 
gardless of whether they moved away or stayed in place?” 

The housing development perspective relates to varying groups of persons who were 
living in Jobs-Plus developments at different points in time. Thus, it focuses on a series of con- 
secutive, partly overlapping cohorts of residents. From this perspective, the Jobs-Plus evaluation 
addresses the question “How did the program affect the future conditions in its target develop- 
ments, regardless of who stayed in place, moved away, or moved in?” 

The distinction between these two perspectives is central to any evaluation of a place- 
based initiative, because sample members can enter and leave its target area. For example, stu- 
dents can move into or out of schools that are implementing a whole-school reform; families 
can move into or out of communities that are implementing a health education program; and 
employees can move into or out of firms that are implementing a worker retention program. 1 In 
these ways, mobility drives a conceptual and operational wedge between people and place. 

Several steps were taken when formulating the Jobs-Plus evaluation to account for this 
phenomenon. The first step was to define a target group that would provide the core sample for 
estimates of program impacts from the individual perspective. This group, which is referred to as 
the “1998 cohort,” comprises all working-age, nondisabled persons who lived in a Jobs-Plus de- 
velopment or a comparison development in October 1998 — when the program was launched. 

'Bloom, Ham, Melton, and O'Brien (2001) address this issue in the context of a whole-school reform. 
Murray et al. (1994) address it in the context of a community health program. MDRC is currently addressing 
the issue in the context of a firm-based retention program for low-wage workers; no reports on this project are 
yet available. 
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The 1998 cohort includes all residents who were between 21 and 61 years of age at the time and 
were not listed as disabled by their local housing authority. By tracking these persons backward in 
time through data from state and local administrative records, it was possible to learn about their 
work and welfare experiences during the six-year Jobs-Plus baseline period, from 1992 through 
1997. By following them forward in time, it was possible to learn about their work and welfare 
experiences during the six-year Jobs-Plus follow-up period, from 1998 through 2003. 

Because it took several years for Jobs-Plus to implement its three program components 
— employment services, financial incentives, and community support for work — many of the 
1998 cohort members had moved away from their housing developments before this was ac- 
complished. For example, between 1 7 percent (in Los Angeles) and 48 percent (in Dayton) had 
moved away within two years. Hence, their exposure to the program was limited, and estimates 
of its impacts for them might understate the effects of exposure to the full program. 

To address this issue, the analysis of program effects was repeated for the cohort of 
working-age, nondisabled adults who were living in a Jobs-Plus development or a comparison 
development in October 2000 (the “2000 cohort”). Some members of this cohort were also 
members of the 1998 cohort, but many had moved into their Jobs-Plus development or com- 
parison development between 1998 and 2000. The main reason for using the 2000 cohort to 
estimate the effects of Jobs-Plus is that these residents had an opportunity to be exposed to the 
complete program (at least in sites that completed the program’s implementation). The primary 
weakness of using the 2000 cohort is that Jobs-Plus might have influenced this group’ s compo- 
sition through effects on residents’ mobility. 2 Hence, findings for the 2000 cohort are used 
mainly to test the sensitivity of findings for the 1998 cohort. 

The next step in the formulation of the analysis of program effects was to make it op- 
erational from the development perspective. To do so, an extensive search was conducted of 
local housing authority records to identify all working-age, nondisabled residents who lived in 
each Jobs-Plus development and comparison development during each calendar quarter from 
1992 through 2003. 3 To make this massive data acquisition process more manageable, it pro- 
ceeded as follows. First, a complete list was developed of working-age, nondisabled residents 
for October of each year. 4 Everyone on each October list was considered to be a fourth-quarter 
resident of the housing development for that year. In addition, everyone on each October list 



2 In other words, the composition of the 2000 cohort could be endogenous. 

3 To simplify the identification of residents in the sample during early baseline years (for which data from 
local public housing authorities were especially limited and difficult to obtain), only persons were included 
who lived in a household headed by a working-age, nondisabled person. Thus, a few able-bodied, working-age 
persons who lived in a household headed by an elderly person or by a person listed as disabled by the public 
housing authority were omitted from the development-level impact analyses. 

4 Persons who were residents for more than one of these years were on more than one October list. 
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was assumed to be a resident during the preceding quarter (Quarter 3 of the same year) and the 
following quarter (Quarter 1 of the next year). To identify residents for the second calendar 
quarter of each year, the “difference was split” between the two October lists that bracket the 
quarter. 5 Administrative records from state and local agencies were then used to obtain data on 
the work and welfare experiences of each quarter-specific group of residents. 

From both the individual and the development perspectives, estimates of the effects of 
Jobs-Plus are reported for each site in the study. These site-specific estimates provide six repli- 
cations of the analysis, which strengthen it considerably by helping to rule out alternative ex- 
planations of program effects that are observed. If a consistent pattern of positive effects is ob- 
served across most sites (which is the case), it is most plausible that Jobs-Plus — rather than 
separate, idiosyncratic, site-specific factors — caused these effects. 

A second important benefit of having site-specific findings is that they make it possible 
to compare program effects for each site with what is learned from field research about how the 
Jobs-Plus program unfolded, what participation in the program was like, what factors influ- 
enced the program’ s evolution and performance, and what the situation was like at comparison 
developments. As with most demonstration programs, the different Jobs-Plus sites had very dif- 
ferent stories to tell in these regards. 

For each site and perspective, estimates of the effects of Jobs-Plus are reported separately 
for each follow-up year. This makes it possible to compare how impacts evolved over time with 
how the program unfolded over time — which is especially important, given the multiyear gesta- 
tion period of Jobs-Plus and its highly variable site-specific implementation experience. 

For each site, perspective, and follow-up year, estimates of the effects of Jobs-Plus are 
reported for two measures of work (earnings and employment rates) and two measures of wel- 
fare (benefit payments and benefit receipt rates). Historical data on earnings and employment 
rates were obtained from the administrative records of state Unemployment Insurance agencies. 
These data are reported quarterly by employers to all states and cover well over 90 percent of 
the jobs in the formal labor market. 6 

From these data, a time series was created of average (mean) quarterly earnings and 
quarterly employment rates for each program group and comparison group. Earnings are re- 
ported in constant 2003 dollars, and quarterly employment rates are defined as the percentage of 
sample members who had at least one dollar of reported earnings in a given quarter. Estimates 
of impacts on average annual earnings provide a comprehensive summary of the effects of Jobs- 



5 The analysis includes all persons who were on both of the October lists plus a random sample of half the 
persons who were only on the earlier list and half the persons who were only on the later list. 

6 Kornfeld and Bloom, 1999. 



218 




Plus on how many sample members were employed, how much they worked when they were 
employed, and how much they were paid for the work that they did. Estimates of the effects of 
Jobs-Plus on average quarterly employment rates for each year indicate how the program 
changed the mix of work and nonwork. 

The welfare-related measures that were constructed are average (mean) annual benefits 
payments received and average quarterly benefit receipt rates. These measures are based on 
monthly welfare receipt data (for AFDC or TANF) obtained from the administrative records of 
state and local agencies. This type of information has been used to evaluate many welfare and 
employment programs. Estimates of program effects on average annual welfare benefits are 
reported in constant 2003 dollars. Estimates of program effects on average quarterly welfare 
receipt rates are reported as the percentage of sample members who received at least one dollar 
of benefits during a given quarter. 7 

Estimates of the effects of Jobs-Plus on average annual benefits received provide a 
comprehensive summary of the program’ s effects on how many target group members relied on 
welfare during a year, how long they relied on it when they did, and how much they received 
when they were drawing benefits. Estimates of the effects of Jobs-Plus on mean quarterly wel- 
fare receipt rates indicate how the program changed the mix of welfare and nonwelfare. 



The Estimation Approach 

For each perspective, site, and outcome measure, a “short comparative interrupted time- 
series analysis” was used to estimate the effect of Jobs-Plus. 8 This rather lengthy name accu- 
rately reflects the approach’ s essential ingredients, which are outlined below. 

Comparing Outcome Levels 

The starting point of the analysis is a graph of mean quarterly outcomes for the Jobs- 
Plus program group and its comparison group. Panel A of Appendix Figure D. 1 illustrates such 
a graph for quarterly earnings in Los Angeles. This graph tells virtually the whole story of the 
analysis of program effects because “what you see is what you get.” 



7 Monthly welfare data were collapsed into quarterly welfare receipt rates to simplify this analysis and to 
make it comparable to the analysis of quarterly employment rates. 

8 Short interrupted time-series analysis applies methods like random-growth models that are used to ana- 
lyze panel data (Greene, 1999). Shadish, Cook, and Campbell (2002, pp. 198-203) distinguish short interrupted 
time-series analysis from traditional time-series analyses that estimate ARIMA models with transfer functions 
from lengthy time-series data. Bloom (1999) explores some of the properties of short interrupted time-series 
analysis of student achievement. Bloom (2003) provides a detailed discussion of the approach. 
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The Jobs-Plus Demonstration 
Appendix Figure D.l 

Average Quarterly Earnings and Differences in Average Quarterly Earnings 
for the Jobs-Plus Group and Its Comparison Group 
in Los Angeles (1998 Cohort) 



A. Earnings 




CN <N CN <N 



◄ Baseline Follow-Up ► 



B. Differences in Earnings 




SOURCES: MDRC calculations using data from housing authority tenant (50058) records and state 
Unemployment Insurance (UI) wage records. 

NOTES: The 1998 cohort includes all residents of a Jobs-Plus development or a comparison 
development in October 1998 who were between 21 and 61 years old and were not listed as disabled 
by their public housing authority. 

DD 01ql is the deviation in Quarter lof 2001 from the baseline mean difference. 
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First note the baseline trends. As can be seen, average earnings for both the Jobs-Plus 
group and its comparison group rose steadily throughout the baseline period. This pattern was 
observed at all sites in the study. Because the baseline period was a time of sustained and sub- 
stantial economic growth, these earnings trends are consistent with the hypothesis that a “rising 
tide lifts all for most] boats.” 

A second key feature of the baseline trends is their remarkable match. Not only are the 
groups’ initial baseline earnings levels similar, but so are their patterns of change over time, 
even in the midst of a dynamic period of rapid economic growth. This lengthy and close base- 
line match held for earnings and employment rates at all six sites in the study. Thus, if subse- 
quent differences in earnings materialized, it would be plausible to infer that they were caused 
by Jobs-Plus, not by preexisting differences. From the individual perspective, the baseline 
match for earnings and employment outcomes for the Jobs-Plus group and its comparison group 
was excellent at all six sites. 

The match was also quite good for work outcomes from the development perspective, 
although these findings contained more random error over time (“noise”) because they represent 
changes in people as well as changes in outcomes for a given group. In addition, there were 
some anomalies in the development-level matches at several sites during the earliest baseline 
years. This probably represents error in the identification of early residents, since the data used 
for identifying residents were maintained less well by local public housing authorities and were 
more difficult to obtain for early years than for later years. Because of these anomalies, esti- 
mates of the effects of Jobs-Plus on development-level earnings and employment rates were 
based on a truncated baseline period that began in 1994 (described below). Sensitivity tests that 
reestimated these program effects using data for the full baseline period indicate that truncating 
the baseline did not change the basic story of the findings. 

The pattern of baseline outcomes was very different for welfare outcomes, and the base- 
line match for these outcomes was much less good than the match for earnings and employ- 
ment. Thus, as described in Chapters 4 and 5, analyses of the effects of Jobs-Plus on welfare 
outcomes were purely graphical and did not use the statistical method described in this appendix 
for estimating program effects on earnings and employment rates. 

Now consider the follow-up experiences illustrated by Panel A of Appendix Figure D. 1 . 
First note that mean quarterly earnings for the Jobs-Plus group and the comparison group re- 
mained very similar during the first two years after the program was launched in Los Angeles. 
This is consistent with the fact that it took several years to implement all three core components 
of Jobs-Plus. Soon thereafter, however, earnings for the Jobs-Plus group jumped well above 
those for the comparison group. This large relative improvement was sustained throughout the 
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remainder of the follow-up period. Given the high-quality baseline match, this subsequent large 
and sustained follow-up divergence strongly suggests that Jobs-Plus caused a pronounced and 
prolonged increase in the earnings of its target group members from this site. 

Tracking Outcome Differences 

Panel B of Appendix Figure D.l presents an alternative way to view the story in Panel 
A. This view, which is the basis for the present statistical analysis of the effects of Jobs-Plus, is 
expressed in terms of quarterly differences in mean earnings for the program group and com- 
parison group in Los Angeles. Starting in 1992 and continuing through 2003, this difference — 
the vertical distance between the earnings levels in Panel A — was computed for each quarter. 
These quarterly differences were then graphed in Panel B. 

When examining the figure, it is useful to start with the baseline period. Note that quar- 
terly differences in mean earnings fluctuate randomly around a constant value — the overall 
baseline mean, which is illustrated by a dashed horizontal line. This illustrates that the baseline 
earnings trends for the Jobs-Plus group and comparison group are approximately parallel to 
each other. 9 Next note that the mean value of the baseline earnings differences is close to zero 
and that quarterly values fluctuate slightly around this mean. This illustrates that the two base- 
line earnings trends are consistently close together. 

Now consider the follow-up period. During the first two follow-up years, quarterly 
earnings differences did not vary substantially or systematically from their baseline mean. This 
indicates that Jobs-Plus had little or no impact on earnings during the first two follow-up years 
— when the program was first being implemented. Subsequently, however, the earnings differ- 
ence jumped well above its baseline mean value and remained there for the rest of the follow-up 
period. This indicates that Jobs-Plus had a large, sustained positive impact on earnings after the 
program was fully implemented. 

Panel B illustrates graphically how statistical estimates of the effect of Jobs-Plus were 
obtained for a given follow-up quarter. This estimate is equal to the observed difference be- 
tween the earnings difference for that quarter and the mean earnings difference for the baseline 
period. This “difference in differences” for Quarter 1 of 2001 is illustrated by the vertical dis- 
tance DD mq i in the figure. 

A more general way to formulate the situation in Panel B is to think of it as an inter- 
rupted time-series analysis of quarterly earnings differences. From the observed baseline eam- 



9 One benefit of viewing the situation through outcome differences is that, regardless of the shapes of the 
groups' baseline trends, their differences will vary randomly around a constant value if their trends are parallel. 
This point is demonstrated below; see “The Statistical Method." 
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ings differences, one can model the baseline trend in these differences. This trend was a random 
fluctuation around a constant baseline mean for all Jobs-Plus sites and was modeled as such. If, 
instead, the baseline trend had been one of systematically rising earnings differences or system- 
atically falling earnings differences, then it could have been modeled as a linear or nonlinear 
function of time. This model is the first part of the interrupted time-series analysis. Note that 
before using a baseline mean, model sensitivity tests were conducted to determine whether there 
was a linear or quadratic trend or seasonality in the baseline earnings differences for the Jobs- 
Plus sites (see “The Statistical Method,” on page 232); there were none for earnings and em- 
ployment rates for the 1998 cohort. 

The second part of the analysis projects the baseline model into the follow-up period to 
predict what quarterly earnings differences would have been if Jobs-Plus had not been 
launched. The baseline mean difference was used to estimate this counterfactual difference for 
each site, as is illustrated in Panel B by the extension of the dashed horizontal line into the fol- 
low-up period. 

The third and final part of the interrupted time-series analysis is to compute the deviation 
from the projected baseline trend for each follow-up quarter. These deviations reflect how Jobs- 
Plus interrupted the time series of quarterly earnings differences; hence, the name “interrupted 
time-series analysis.” The interruption provides estimates of the effect of Job-Plus. Hence, the 
procedure used to estimate the effect of Jobs-Plus is based on “deviations in differences.” 10 

Specifying the Statistical Model 

Equations D. 1 and D.2, which specify a baseline mean model for an interrupted time se- 
ries of quarterly outcome differences, represent the statistical model used to estimate the quar- 
terly effects of Jobs-Plus. The dependent variable in Equation D.l is the program and compari- 
son outcome difference, A y r , for each quarter. The independent variables are a series of binary 



10 Earlier presentations of the analysis of program effects that was planned for Jobs-Plus (Bloom and Ric- 
cio, 2002; Bloom, 1996) are formulated in a way that is different from but equivalent to the current formulation 
under a broad range of conditions. The earlier formulation is based on two separate interrupted time-series 
analyses of mean quarterly earnings levels. The first analysis — for the Jobs-Plus group — measures its devia- 
tion from its baseline trend for each follow-up quarter. This represents how the earnings trajectory for the pro- 
gram group changed (or did not) after Jobs-Plus was launched. The second analysis — for the comparison 
group — measures its deviation from its baseline trend for each follow-up quarter. This predicts how the earn- 
ings trajectory for the Jobs-Plus group would (or would not) have changed if Jobs-Plus had not been launched. 
The effects of Jobs-Plus are estimated as the difference for each follow-up quarter between its deviations from 
trend for the Jobs-Plus group and for the comparison group. This “difference in deviations” represents the 
change in mean earnings that was caused by Jobs-Plus. If the functional forms of the underlying trends for the 
two groups are the same and they are linear in their parameters, then the two formulations yield the same im- 
pact estimate. In other words, the deviation in differences equals the difference in deviations. 
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(zero/one) indicators, F mt , which identify each follow-up quarter. The error term, A et , has a 
first-order autoregressive distribution (Equation D.2). 



A K = A c/ + X Ag,„F„„ + A e , 

m 



(D.l) 



and 



Ae,- pAe,-i + A u, 



(D.2) 



where: 

A Yi = the difference in mean earnings for the program group and comparison 
group in quarter t 

F mt = one if quarter t is follow-up quarter m and zero otherwise 

A et, A e t - 1 = error terms with a first-order autoregressive distribution 

A D, = a random error term 

The intercept, A a , in Equation D.l equals the baseline mean outcome difference for 
the program group and comparison group, which is represented by the dashed horizontal line in 
Panel B. The coefficients, A g m , equal the deviation of the outcome difference for each follow- 
up quarter from the baseline mean outcome difference, which is represented by the vertical dis- 
tance DDoiqi for the first quarter of 2001 in Panel B. The value of this coefficient equals the im- 
pact estimate for the follow-up quarter that it represents. The standard error for this coefficient is 
the standard error for the quarterly impact estimate. 

The preceding model was estimated for each Jobs-Plus site and outcome using meas- 
ures of quarterly differences in mean earnings and employment rates. This type of analysis was 
conducted from the perspective of public housing residents and from the perspective of public 
housing developments. 

Estimating Quarterly Program Effects 

Using maximum likelihood methods to estimate the statistical model in Equations D.l 
and D.2 from data for the earnings differences in Panel B of Appendix Figure D. 1 yields the 
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quarterly findings for program effects in Appendix Table D.l. 11 First consider the baseline find- 
ings. The estimated value of $19 for the intercept in Equation D.l equals the mean quarterly 
earnings difference for the baseline period. This is an extremely small difference relative to the 
earnings levels for the two groups, which range from about $1,000 to $1,500 per quarter. The 
estimated value of $36 for the standard error of the intercept indicates that the baseline earnings 
difference for the two groups did not vary much from quarter to quarter. These two findings 
from the statistical analysis indicate not only that the earnings levels of the two groups in the 
example were close together, on average, but also that they were consistently close together. 

Now consider the findings for the follow-up period. The estimated coefficient for each 
follow-up quarter equals its observed deviation from the mean earnings difference for the baseline 
period. This is the measure of the effect of Jobs-Plus on earnings for each quarter. The standard 
error for this estimate is related to its statistical significance. Note that quarterly program effect 
estimates for the first two follow-up years are small both absolutely and relative to their standard 
errors. Hence, they are inconsequential and not at all statistically significant. Subsequently, how- 
ever, these estimates are much larger in absolute terms and generally between two and four times 
the size of their standard errors. Hence, they are consequential and highly statistically significant. 
These findings provide a quantitative way to tell the qualitative story in the graphs. 

Lastly, note the findings for the first-order serial correlation coefficient. Its value of -0.61 
is fairly large. And the fact that it is many times the size of its standard error of 0.18 indicates that 
it is highly statistically significant. Thus, it represents an important element of the model. 

Computing Annual Program Effects 

To reduce the number of findings to be reported and interpreted, the quarterly estimates 
in Appendix Table D.l were cumulated into the annual estimates in Appendix Table D.2. The 
first column in the new table lists estimates of the effects of Jobs-Plus on average earnings for 
each follow-up year. These were obtained by summing the relevant quarterly follow-up coeffi- 
cients. 12 Thus, for example, the annual estimate in Appendix Table D.2 for 1998 equals the sum 
of the quarterly follow-up coefficients in Appendix Table D.l for 1998 (any apparent inconsis- 
tencies are due to rounding). The second column in Appendix Table D.2 lists the estimated 
standard error for each annual estimate based on the estimated variances and covariances for its 
quarterly follow-up coefficients — as described below in this appendix. The third column in the 
table lists the initial p-value implied by each impact estimate and its standard error. The fourth 



1 'These estimates were obtained using SAS PROC AUTOREG. 

I2 To compute annual program effects on earnings, the quarterly estimates were summed. To compute an- 
nual effects on employment rates, the quarterly estimates were averaged. Standard errors of the annual esti- 
mates were calculated accordingly. 
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The Jobs-Plus Demonstration 



Appendix Table D.l 

Parameters of the Outcome Difference Model 
for the Example in Appendix Figure D.l 



Parameter 


Point Estimate 
(dollars) 


Standard Error 
(dollars) 


Intercept 


19 


36 


Coefficient for the follow-up quarter 


1998 Q1 


7 


78 


1998 Q2 


-30 


93 


1998 Q3 


-69 


98 


1998 Q4 


144 


99 


1999 Q1 


-19 


100 


1999 Q2 


-42 


100 


1999 Q3 


-158 


100 


1999 Q4 


-104 


100 


2000 Q1 


39 


100 


2000 Q2 


188 


100 


2000 Q3 


152 


100 


2000 Q4 


169 


100 


2001 Q1 


180 


100 


2001 Q2 


270 


100 


2001 Q3 


338 


100 


2001 Q4 


378 


100 


2002 Q1 


383 


100 


2002 Q2 


314 


100 


2002 Q3 


279 


100 


2002 Q4 


440 


100 


2003 Q1 


237 


100 


2003 Q2 


369 


100 


2003 Q3 


356 


100 


2003 Q4 


390 


100 


Serial correlation coefficient (first order) 


-0.61 


0.18 
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The Jobs-Plus Demonstration 



Appendix Table D.2 

Annual Impact Estimates of Jobs-Plus 
for the Example in Appendix Figure D.l 



Follow-Up Year 


Point Estimate 


Standard Error Initial P- Value 


Bonferroni -Adjusted P- Value 


1998 


51 


290.71 


0.861 


0.861 


1999 


-323 


321.62 


0.327 


0.654 


2000 


548 


321.63 


0.103 


0.308 


2001 


1,165 


321.57 


0.002 


0.006 


2002 


1,415 


321.55 


0.000 


0.001 


2003 


1,351 


321.55 


0.000 


0.002 


2000-2003 


1,120 


228.00 


0.000 





NOTE: Rounding obscures the relationships between the initial and adjusted p-values. 



and final column lists each p-value after it was adjusted using a “layered Bonferroni” approach 
to account for the “multiple testing” problem created by the presence of six annual impact esti- 
mates (described below). For assessing the statistical significance of these annual impact esti- 
mates, only the adjusted p-values are reported. 

The findings in Appendix Table D.2 tell a striking story about program effects on earn- 
ings that are delayed for two years but that then become substantial, sustained, and statistically 
significant. This is precisely the impression conveyed by the earnings graphs in Appendix Fig- 
ure D.l. A summary of these findings for the period from 2000 to 2003, after Jobs-Plus was 
implemented, was computed by averaging the annual findings for this period and computing the 
standard error accordingly. 13 This finding indicates that, on average, for the 1998 cohort in Los 
Angeles, Jobs-Plus increased average earnings by $1,120 per year during the period. 

Choosing Comparison Groups 

A final key feature of the analysis of Jobs-Plus effects is how comparison groups were 
chosen. This process had three phases. The first phase involved constructing a matched group of 
two or three candidate housing developments from each Jobs-Plus site and using computer- 



13 The standard error of the overall average accounted for the interdependencies among the quarterly esti- 
mates for the period. 
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generated random numbers to choose one program development and one or two potential com- 
parison developments from each pair or triplet. 14 This process made it impossible to “stack the 
deck” by consciously or inadvertently choosing program developments that were more likely or 
less likely than others to improve on their own. Four sites (Baltimore, Chattanooga, Dayton, and 
St. Paul) each had two potential comparison developments, and two sites (Los Angeles and Se- 
attle) each had one potential comparison development. 

The next phase of the process was to further check each comparison development to 
ensure that it was similar enough to the Jobs-Plus development at baseline to provide an ade- 
quate benchmark for making future comparisons. This assessment used the results of a base- 
line survey that was administered to a sample of household heads from five sites in 1998 and 
one site (St. Paul) in 1999. These results indicated that the characteristics of residents and de- 
velopments were quite similar for each program group and its comparison group. Hence, at 
baseline, all potential comparison developments were judged to provide an adequate bench- 
mark for their Jobs-Plus development. 

The last phase of the selection process was to ensure that nothing had occurred in a 
comparison development during the follow-up period that would invalidate its use for estimat- 
ing the effects of Jobs-Plus. This assessment, which was based on field research, scrutinized 
each local situation in a systematic search for any compelling reasons to believe that a compari- 
son development would not provide a “fair test” of Jobs-Plus. Potential reasons included, 
among others: (1) a major disruption in the public housing development (like that caused by a 
HOPE VI renovation that displaces many residents), (2) a major economic shock to the local 
low-wage labor market (like that caused by the opening or closing of a large employer of low- 
wage workers in the immediate vicinity), or (3) a major competing employment program oper- 
ating in or through the comparison development that would compromise its ability to emulate 
the counterfactual for Jobs-Plus. 

Although not stated explicitly in a written protocol, the decision rules for choosing 
comparison developments were established at the outset of the Jobs-Plus project. Only if a 
compelling reason were found for dropping a comparison development would this be done. If 
this occurred for a site with two initial comparison developments, the analysis of Jobs-Plus’ s 
effects would be based on outcomes for the remaining development. If this occurred for a site 
with one initial comparison development, that site would have no analysis of the effects of Jobs- 



14 As noted in Chapter 1, the ability for MDRC to make this choice randomly was an eligibility require- 
ment for each site, and all 15 cities that became semifinalists accepted this requirement. Although explaining 
the need for the requirement took considerable time and effort, it was generally recognized that Jobs-Plus rep- 
resented a scarce resource whose allocation by lottery was quite ethical. Furthermore, anecdotal evidence sug- 
gests that at least some local residents felt that random selection of the participating housing development 
would be fairer than selection by the public housing authority — which was not always trusted. 



228 




Plus. For sites with two comparison developments that remained in contention, comparison 
group outcomes would be computed as an equally weighted average of those for the two com- 
parison developments. 

Only one “smoking gun” was identified at a comparison development, and so only one 
was considered to be invalid and thus excluded from the analysis. This occurred in St. Paul, 
where, at one comparison development, a major employment program plus a series of related 
social services described in Appendix Box D.l were instituted on-site in response to a tragic 
event that had occurred there. This left three sites (Baltimore, Chattanooga, and Dayton) with 
two valid comparison developments each and three sites (Los Angeles, St. Paul, and Seattle) 
with one valid comparison development each. 

Combining Sites 

The example in this appendix illustrates how the effects of Jobs-Plus were estimated for 
a single site. To summarize findings across sites, it was necessary to combine their data. This 
was done in a way that reflected the broad variation in the implementation experiences of the 
sites, maintained an ability to graphically examine the combined findings for the baseline and 
follow-up periods, and was as simple as possible. To properly reflect implementation experi- 
ences, the findings for sites were combined two different ways. First, all six sites were com- 
bined to address the question “What is the average effect of Jobs-Plus for all versions of the 
program that are implemented?” Second, the three “stronger implementation sites” — Dayton, 
Los Angeles, and St. Paul — were combined to address the question “What is the average effect 
of Jobs-Plus when it is implemented relatively well?” Findings were presented both ways to 
address both questions. 

To facilitate a graphical analysis of the time series of combined outcomes and to keep 
the analysis as simple as possible, the quarterly data for sites were combined by taking their 
mean across sites, with each site being weighted equally. In this way, a composite time series 
was created for the Jobs-Plus groups and for the comparison groups. The combined analysis of 
the effects of Jobs-Plus was conducted from the two composite time series. 

Conducting Sensitivity Tests 

Several tests were conducted to see how sensitive estimates of the effects of Jobs-Plus 
were to decisions about how they were obtained. One test assessed the importance of excluding 
the invalid comparison development in St. Paul, which had an extensive on-site employment 
program. Estimates in the present report exclude that development and indicate that Jobs-Plus 
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Appendix Box D.1 



Employment-Related Services 
at McDonough Homes in St. Paul 

Working residents at McDonough Homes have a wide array of services available to help 
them secure and retain employment and access benefits for working-poor families. Most of 
these services can be accessed through McDonough’s large and spacious community center, 
which was completed in 1997 and houses multiple office suites, rooms for meetings and 
classes, and athletic facilities. 

Employment services at the center are provided primarily by the Support for Training and Em- 
ployment Program (STEP) of St. Paul Public Schools. STEP offers vocational training, job 
preparation (for example, help with resume writing, filling out applications, and job inter- 
views), job search assistance, and referrals to off-site training and educational opportunities. 
These services are available on both an individualized and group basis. STEP also offers Eng- 
lish as a Second Language (ESL) and adult basic education (ABE) classes. Two STEP em- 
ployment counselors are available four days a week, as are two part-time bilingual assistants, 
two part-time computer teachers, and three ESL teachers. Residents also have the use of a 
computer room with ten computers and a job resource room with two phone cubicles, more 
computers, and information about job opportunities, job search, and education and training 
programs. The computers have Office 2000 software, Microsoft 98, a resume-maker, and ac- 
cess to the Internet and job listings on the Minnesota Career Infoimation System (MCIS). 

Welfare recipients at McDonough Homes can also get employment assistance from Evenstart, 
which is funded by the Minnesota Family Investment Program (MFIP), the state’s welfare pro- 
gram. Evenstart offers job preparation and family counseling to MFIP clients who are having 
trouble transitioning to work. Welfare recipients can also have their cases assigned to one of 
two MFIP employment counselors who are stationed conveniently at the center. MFIP coun- 
selors and Evenstart staff are working together to help stabilize the employment of hard-to- 
employ welfare recipients facing lifetime limits on their benefits. Finally, the Sisters of St. Jo- 
seph operate MORE (McDonough Organization for Respect and Equality) in an adjacent 
building, which offers ESL and citizenship classes as well advocacy services. 

For working parents who need help caring for their children during the workday, the com- 
munity center offers options for all age levels. Head Start has three classrooms for preschool- 
ers, and Growing Places provides all-day care for children ages 2 V 2 to 5 years old. Rec 
Check provides after-school care for school-age children. The center also qualifies as a full- 
service recreation center of the St. Paul Parks and Recreation Department. Youths have ac- 
cess to two full-size gyms, including several basketball courts. There arc 4-H programs for 
children ages 13 and younger and for teenagers. The center has a staff of teen counselors, an 
active Boy Scout troop, and a Youth Cares girls’ group; the sheriff department’s Youth Lit- 
eracy Program provides after-school tutoring. 

-■{- 

This account was written by Linda Yuriko Kato of MDRC. 

(continued) 
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Appendix Box D.1 (continued) 



Residents of McDonough Homes also have convenient access at the community center to 
benefits that can help the working poor sustain their families on low-wage jobs without em- 
ployer-provided benefits. For instance, qualified families can get food vouchers at the office 
of the Women, Infants, and Children (WIC) program. And all residents can get medical and 
mental health care from the West Side Health Clinic, which is a full-scale clinic occupying a 
sizable area in the center. The clinic staff include several full-time public health nurses as 
well as a physician who is available four days a week and a social worker who has a counsel- 
ing degree and visits once a week. 

It is also important to emphasize that the community center at McDonough Homes is unusual 
in having a general manager and overall coordinator of its services. The center benefits im- 
mensely from the administrative direction and coordination of its manager, who has held this 
position since the center opened. He is the person who has the “big picture” of the residents’ 
needs and the availability of services at the center to address them, as well as extensive 
knowledge of and relationships with other service providers in the St. Paul-Minneapolis area. 
The manager describes himself as a “social services networker” who operates like a “spider” 
in building cooperative links among the various service providers that are colocated at 
McDonough Homes, and between them and other providers in the area. His coordinating role 
helps to promote service integration and reduce duplication of efforts at the center. At the 
same time, he acts as a liaison between the on-site providers and the housing management 
office. For instance, as a housing authority employee, he is privy to otherwise difficult-to- 
acquire infoimation about residents’ job losses or domestic problems that can undercut their 
employment, which the residents and security police report to the management office and 
which he then relays to the appropriate providers to address. And the residents usually turn to 
him first as the center’s familiar and hoisted point person to steer them in the right direction for 
help. “Part of my job,” he says, “is hying to reduce the number of hoops people have to jump 
through, and piggybacking some services onto the same caseworker.” The manager works with 
the residents, the providers, and the management office to regularly assess the community’s 
needs and to ensure that the center’s services are appropriate and responsive to those needs. 

Finally, McDonough Homes offers residents the security of being able to go to work know- 
ing that their homes and families will be safe during their absence. McDonough is the head- 
quarters of ACOP (A Community Outreach Policing program), which is funded by the hous- 
ing authority in St. Paul to promote safety in its developments. The sizable and prominent 
presence of the police at McDonough — with a fleet of patrol cars parked in front of the 
community center — has enhanced the peace and security of the development to the point 
where its crime rates arc comparable to those of St. Paul’s affluent neighborhoods. 
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increased average annual earnings from 2000 to 2003 for the 1998 cohort in St. Paul by $1,492 
per year. Alternative estimates that include the development indicate a $909 gain. Both esti- 
mates suggest that the program produced large earnings gains, and both gains are highly statisti- 
cally significant. The implications of this difference for estimates of the effects of Jobs-Plus for 
the combined sample of stronger implementation sites are even less pronounced. The estimate 
of program effects on average annual earnings from 2000 to 2003 was $1,141 without the inva- 
lid comparison development and $1,013 with it. 

A second sensitivity test of the comparison group strategy that was conducted was to 
repeat estimates of the effects for each site that had two valid comparison developments — Bal- 
timore, Chattanooga, and Dayton — using one of its comparison developments at a time. For 
Baltimore and Chattanooga, these tests produced the same results for both comparison devel- 
opments: No effects of Jobs-Plus were found using either one. For Dayton, estimates based on 
one comparison development indicated very large positive program effects, whereas estimates 
based on the other comparison development indicated little or no effects. As planned from the 
outset of the project, however, the mean for the two valid comparison developments was used to 
estimate the effects of Jobs-Plus for these three sites, since doing so uses all the best information 
that is available. 

A third sensitivity test was conducted to assess the influence of truncating the early part 
of the baseline earnings history for estimates of program effects on housing developments — to 
account for some aberrations that exist in the early data for Los Angeles, in particular, and for 
Baltimore, to a lesser extent. To do so, estimates of program effects were obtained both for the 
truncated baseline period, from 1994 to 1997 (which is used for findings in the present report), 
and for the full baseline period, from 1992 to 1997, in those sites where the development-level 
analysis was conducted and full baseline data exist: Baltimore, Dayton, and Los Angeles. Esti- 
mates of program effects on average earnings from 2000 to 2003 for the truncated baseline ver- 
sus the full baseline, respectively, were -$489 versus -$306 for Baltimore; $584 versus $367 
for Dayton; and $1,581 versus $2,161 for Los Angeles. Hence, only in Los Angeles did truncat- 
ing the baseline make much difference, and doing so in this case reduced the estimated effect of 
Jobs-Plus on development-level earnings. 

The Statistical Method 

This final section describes in somewhat more detail the statistical method used to esti- 
mate the effects of Jobs-Plus. The method: 

• Has a simple graphical interpretation 

• Focuses on the effects of Jobs-Plus for each follow-up year separately, which 
is a transparent and flexible way to tell how the story unfolds 
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• Eliminates the need to specify functional forms for outcome patterns during 
the follow-up period and reduces the need to specify functional forms for 
baseline outcome patterns 

• Takes advantage of the remarkable degree to which quarterly baseline out- 
comes for each site’ s Jobs-Plus group and comparison group are consistently 
close together 

• Can be derived from a wide range of multilevel models that represent the 
time paths of outcomes for individuals nested within groups 

• Accounts for the multiple testing problem produced by estimating a separate 
program effect for each follow-up year 

• Accounts for serial correlation that can exist among individual outcomes 
over time 

Developing the Method 

The method can be developed in a series of six steps: 

1 . Specify a fairly general model of individual quarterly outcomes. 

2. Aggregate the individual model into one for the mean quarterly outcomes of 
a Jobs-Plus group or a comparison group. 

3. Subtract the mean quarterly outcome model for the comparison group from 
that for the Jobs-Plus group to produce a model of their outcome differences. 

4. Simplify the model of quarterly outcome differences to one with a constant 
value, which applies whenever the baseline trends of the Jobs-Plus group and 
comparison group are parallel to each other. 

5. Cumulate estimates of quarterly program effects into estimates of annual 
program effects to reduce the number of estimates. 

6. Adjust the statistical significance level of each annual impact estimate to ac- 
count for the fact that there are six of them. 

Step 1 : Specify the Individual Outcome Model 

Start with a model of quarterly outcomes for an individual i. This model, presented in 
Equations D.3 and D.4, comprises a quadratic, seasonal, and serially correlated baseline trend 
plus a series of quarterly deviations from this trend for a multiyear follow-up period. The dis- 
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cussion which follows holds for other functional forms that are linear in their parameters, such 
as higher-order polynomials. 

Yu = ci, + b,t + Ci t + Y, dkSkt+y', gmiFm+ett (D.3) 

k m 



and 

e„ = pen, - 1 ) + Vi, (D.4) 

where: 

Y it = the outcome for individual i in quarter t 

t = a counter for time that increments by a value of one for each calendar quarter 

Stt = one if quarter t is in season k and zero otherwise 

F mt = one if quarter t is the m th follow-up quarter and zero otherwise 

The coefficient g mi represents the individual’s deviation from his or her baseline trend 
during follow-up quarter m. A separate deviation from trend is included for each follow-up quar- 
ter to ensure that no follow-up information is used to estimate baseline parameters. This is because 
an indicator variable for a single data point removes that point from the rest of the analysis. 15 

Step 2: Derive the Group Mean Model 

The corresponding model of mean quarterly outcomes for a group j in Equations D.5 
and D.6 can be derived by taking the mean of the individual models for the group’s members. 

Yj, = ai + bjt + at^ + Y,dkjSk, + 2] + e,, (D.5) 

k m 

and 

e# = peat-n + vj, (D.6) 

where: 

a, , b , Ci , dkj and g mj = the mean intercept, slope, acceleration, seasonality, 

and quarterly follow-up deviation for group j 



15 If, instead, an indicator variable had been specified for each follow-up year, quarterly variation in the 
outcome within follow-up years would have been incorporated into estimates of the baseline parameters. 



234 




Step 3: Derive the Group Difference Model 

A corresponding model of the difference in mean quarterly outcomes in Equations D.7 
and D.8 for a program group and a comparison group can be obtained by taking the difference 
between their mean outcome models. 

A Yt — A ci~\~ Ab t Ac -l- A elk Sk -f A g mFm -T A ct (D.7) 

k m 

and 

Aei = pAet - 1 + Aut (D.8) 

where: 

A a,Ab, Ac,Adk anclA g m = the difference in the mean intercepts, slopes, 

acceleration, seasonality, and quarterly follow-up 
deviations for the two groups 

This finding illustrates that individual models, which are additive in their parameters, 
imply a model for group differences in mean outcomes whose parameters equal the differences 
in the group means of their individual parameter values. 

Step 4: Simplify the Group Difference Model 

The model of program and comparison group quarterly outcome differences can be 
simplified to take advantage of the fact that the baseline trends for the Jobs-Plus group and 
comparison group are approximately parallel to each other. This simplification yields Equations 
D.9 and D.10. 



Ay, = A a +TAg,„F„„ + A e , 


(D.9) 


m 




and 




Ae, = pAe,-, + Aor 


(D.10) 



This model is the core of the statistical analysis of the effects of Jobs-Plus on residents’ 
work. For each outcome and site, tests were conducted to determine whether the model applies 16 by 
(1) visually inspecting the baseline trends for the Jobs-Plus group and comparison group to make 
sure they look parallel and (2) examining the size and statistical significance of the estimated slope, 
acceleration, and seasonality parameters to make sure that they can be omitted. For analyses of eam- 



16 Because of the very large number of estimates involved, it was not feasible to check the validity of the 
simplified model for every combination of subgroup, outcome, and site. Instead, the model was checked for the 
full sample for each major outcome for every site. 
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ings and employment impacts for the 1998 cohort, this was the case at all sites. Therefore, all these 
findings are based on maximum likelihood estimates of Equations D.9 and D. 10. 

Step 5: Cumulate Quarterly Estimates into Annual Estimates 

Estimates of annual program effects were created by summing or averaging estimates 
of quarterly program effects obtained from Step 4 and computing standard errors accordingly. 
For example, the estimate of program effects on mean earnings for follow-up Year 1 (1998) was 
obtained by summing estimates of program effects for the first four follow-up quarters. Hence, 



A gi — A g\ + A gi + A g3 + A gi (D. 11) 

and 



4 4 4 _ _ 

I VAR(Ag„)- I ICOV(Ag,„Ag„) (D.12) 

. m=l n^mm^n 

Correspondingly, the estimate of program effects on mean quarterly employment rates 
for 1998 is obtained by averaging estimates of program effects for the first four follow-up quar- 
ters. (Dividing Equations D.l 1 and D.12 by 4 yields the proper expressions for computing these 
annual estimates and their standard errors.) 

Step 6: Adjust for Multiple Testing 

The statistical significance (p-value) of each annual estimate for a given analysis was 
adjusted to account for the fact that six such estimates are obtained (one for each follow-up 
year). For this purpose, a layered Bonferroni approach was used. This approach multiplies the 
p-value for the annual program effect estimate that is most statistically significant (has the 
smallest p-value) by the number of annual estimates reported for the site and outcome. The 
p-value for the next most significant annual impact is then multiplied by one less than the num- 
ber of annual estimates, and so on. This approach provides a conservative test of the statistical 
significance of each annual estimate. It is conservative in the sense that it may understate “true” 
significance somewhat. Hence, if the adjusted p-value of the impact estimate is statistically sig- 
nificant at conventional levels, this provides strong evidence of statistical significance. But if the 
adjusted p-value is not statistically significant, this could result from the adjustment’s being too 
conservative. The statistical properties of the approach used to account for multiple testing are 
judged to be quite good. 17 



SE(Ag,) = 



n Darlington, 1990. 
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Appendix E 

Supplementary Table and Figures for Chapter 4 




The Jobs-Plus Demonstration 

Appendix Table E.l 



Characteristics of the Current or Most Recent Job 
Among Follow-Up Survey Respondents in the Jobs-Plus Developments 



Characteristic 


Baltimore 


Dayton 


St. Paul 


Among those employed within 
the last 12 months: 


Average hours worked per week 


29.0 


29.4 


35.2 


Distribution of hours worked (%) 


1-19 hours 


18.4 


17.0 


4.7 


20-29 hours 


21.5 


25.5 


16.3 


30-39 hours 


24.7 


21.3 


25.6 


40+ hours 


34.2 


35.5 


53.5 


Average hourly wage ($) 


8.8 


8.6 


10.6 


Distribution of hourly wage (%) 


$l-$5. 15 


6.9 


9.2 


1.2 


$5.15-$7.75 


29.9 


43.3 


13.4 


More than $7.75 


63.2 


47.5 


85.4 


Employer-provided benefits (%) 


Any benefits 


46.1 


60.0 


70.9 


Health plan for self 


37.4 


45.3 


44.2 


Health plan for children 


29.6 


35.0 


29.1 


Paid sick days 


32.9 


28.5 


57.0 


Paid vacation days 


39.4 


47.5 


60.5 


Perceived job-related risks (%) 


Health or safety risk 


32.9 


45.0 


NA 


Poor job security 


24.7 


23.0 


NA 


Constantly changing hours 


50.0 


55.4 


NA 


Sample size 


159 


141 


86 



SOURCES: MDRC calculations using data from Jobs-Plus follow-up surveys. 
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The Jobs-Plus Demonstration 
Appendix Figure E.l 

Average Quarterly Employment Rates for the 
Jobs-Plus Group and Its Comparison Group, by Site (1998 Cohort) 
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Appendix Figure E.l (continued) 




Baseline 



Follow-Up 




Mean Quarterly Employment Rate Mean Quarterly Employment Rate 



Appendix Figure E.l (continued) 
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Q1 2003 




Appendix Figure E.l (continued) 

SOURCES: MDRC calculations using data from housing authority tenant (50058) records and state 
Unemployment Insurance (UI) wage records. 

NOTES: The 1998 cohort includes all residents of a Jobs-Plus development or a comparison 
development in October 1998 who were between 21 and 61 years old and were not listed as disabled by 
their public housing authority. 

“Unemployment Insurance (UI) wage data for the second quarter of 1993 were not available for 
Dayton. 
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The Jobs-Plus Demonstration 



Appendix Figure E.2 

Average Quarterly Earnings and Employment Rates for the 
Jobs-Plus Group (1998 Cohort in Imperial Courts, Los Angeles) 





SOURCES: MDRC calculations using data from housing authority tenant (50058) records and state 
Unemployment Insurance (UI) wage records. 

NOTE: The 1998 cohort includes all residents of a Jobs-Plus development in October 1998 who were 
between 21 and 61 years old and were not listed as disabled by their public housing authority. 
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The Jobs-Plus Demonstration 



Appendix Figure E.3 

Average Quarterly Earnings and Employment Rates for the Jobs-Plus Group and 
Its Comparison Group (Black, Non-Hispanic Women in the 
1998 Cohort from St. Paul) 



A. Earnings 
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SOURCES: MDRC calculations using data from housing authority tenant (50058) records and state 
Unemployment Insurance (UI) wage records. 



NOTE: The 1998 cohort includes all residents of a Jobs-Plus development or a comparison 
development in October 1998 who were between 21 and 61 years old and were not listed as disabled 
by their public housing authority. 
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The Jobs-Plus Demonstration 



Appendix Figure E.4 

Average Quarterly Employment Rates for the Jobs-Plus Group and Its 
Comparison Group, for the Largest Demographic Subgroups in the Stronger 
Implementation Sites (1998 Cohort) 
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Appendix Figure E.4 (continued) 



C. St. Paul - Southeast Asian Women 
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Appendix Figure E.4 (continued) 



E. St. Paul - Southeast Asian Men 
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SOURCES: MDRC calculations using data from housing authority tenant (50058) records and state 
Unemployment Insurance (UI) wage records. 

NOTE: The 1998 cohort includes all residents of a Jobs-Plus development or a comparison 
development in October 1998 who were between 21 and 61 years old and were not listed as disabled by 
their public housing authority. 

“Unemployment Insurance (UI) wage data for the second quarter of 1993 were not available for 
Dayton. 
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Supplementary Figures for Chapter 5 
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Appendix Figure F.l 

Average Quarterly Employment Rates for the Jobs-Plus Group 
and Its Comparison Group, by Site (Development-Level Findings) 
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Appendix Figure F.l (continued) 
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Appendix Figure F.l (continued) 
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SOURCES: MDRC calculations using data from housing authority tenant (50058) records and state 
Unemployment Insurance (UI) wage records. 

NOTES: The development-level samples include all nondisabled residents aged 21 to 61 in each year. 
To facilitate data collection, given the complexity of local housing authority records, only persons 
living in a household headed by a nondisabled resident between the ages of 21 and 61 were included in 
these samples. 

Sample sizes varied from year to year ranging from: 264 to 335 in the program group and 519 to 
666 in the comparison group in Baltimore; 240 to 314 in the program group and 349 to 500 in the 
comparison group in Chattanooga; 256 to 346 in the program group and 400 to 628 in the comparison 
group in Dayton; 356 to 466 in the program group and 294 to 387 in the comparison group in Los 
Angeles; and 192 to 289 in the program group and 149 to 21 1 in the comparison group in St. Paul. 

“Unemployment Insurance (UI) wage data for the second quarter of 1993 were not available for 
Dayton. 
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Appendix Figure F.2 

Average Quarterly Welfare Receipt Rates for the Jobs-Plus Group 
and Its Comparison Group, by Site (Development-Level Findings) 
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Appendix Figure F.2 (continued) 



C. St. Paul 




◄ Baseline 
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SOURCES: MDRC calculations using data from housing authority tenant (50058) records and state 
AFDC/TANF records. 

NOTES: The development-level samples include all nondisabled residents aged 21 to 61 in each year. 
To facilitate data collection, given the complexity of local housing authority records, only persons 
living in a household headed by a nondisabled resident between the ages of 2 1 and 6 1 were included in 
these samples. 

Sample sizes varied from year to year ranging from: 256 to 346 in the program group and 400 to 
628 in the comparison group in Dayton; 356 to 466 in the program group and 294 to 387 in the 
comparison group in Los Angeles; and 192 to 289 in the program group and 149 to 21 1 in the 
comparison group in St. Paul. 
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